Methods and apparatus for data analysis

ABSTRACT

A method and apparatus for data analysis according to various aspects of the present invention is configured to test a set of components and generate test data for the components. A diagnostic system automatically analyzes the test data to identify a characteristic of a component fabrication process by recognizing a pattern in the test data and classifying the pattern using a neural network.

CROSS-REFERENCES TO RELATED APPLICATIONS

The present application is:

-   -   a continuation of U.S. patent application Ser. No. 14/855,591        filed on Sep. 16, 2015, entitled METHODS AND APPARATUS FOR DATA        ANALYSIS, which a continuation of U.S. patent application Ser.        No. 13/044,202 filed on Mar. 9, 2011, entitled METHODS AND        APPARATUS FOR DATA ANALYSIS, which is a continuation-in-part of        U.S. patent application Ser. No. 12/573,415, filed on Oct. 5,        2009, entitled METHODS AND APPARATUS FOR DATA ANALYSIS, which is        a continuation of U.S. patent application Ser. No. 12/021,616,        filed on Jan. 29, 2008, now abandoned, entitled, METHODS AND        APPARATUS FOR DATA ANALYSIS, which is a continuation of U.S.        patent application Ser. No. 11/053,598, filed on Feb. 7, 2005,        now U.S. Pat. No. 7,356,430 entitled METHODS AND APPARATUS FOR        DATA ANALYSIS, which:        -   claims the benefit of U.S. Provisional Patent Application            No. 60/542,459, filed Feb. 6, 2004, entitled EVOLVING NEURAL            NETWORKS USING SWARM INTELLIGENCE FOR BINMAP CLASSIFICATION;        -   claims the benefit of U.S. Provisional Patent Application            No. 60/546,088, filed Feb. 19, 2004, entitled DYNAMIC            OUTLIER ALGORITHM SELECTION FOR QUALITY IMPROVEMENT AND TEST            PROGRAM OPTIMIZATION; and is a continuation-in-part of U.S.            patent application Ser. No. 10/817,750, filed on Apr. 2,            2004, now U.S. Pat. No. 7,395,170 entitled METHODS AND            APPARATUS FOR DATA ANALYSIS, which is a continuation-in-part            of U.S. patent application Ser. No. 10/730,388, filed on            Dec. 7, 2003, now U.S. Pat. No. 7,225,107 entitled METHODS            AND APPARATUS FOR DATA ANALYSIS, which:            -   claims the benefit of U.S. Provisional Patent                Application No. 60/483,003, filed Jun. 27, 2003,                entitled DEVICE INDEPENDENT WAFERMAP ANALYSIS; and is a                continuation-in-part of U.S. patent application Ser. No.                10/367,355, filed on Feb. 14, 2003, now U.S. Pat. No.                7,167,811 entitled METHODS AND APPARATUS FOR DATA                ANALYSIS, which is a continuation-in-part of U.S. patent                application Ser. No. 10/154,627, filed on May 24, 2002,                now U.S. Pat. No. 6,792,373 entitled METHODS AND                APPARATUS FOR SEMICONDUCTOR TESTING, which:                -   claims the benefit of U.S. Provisional Patent                    Application No. 60/293,577, filed May 24, 2001,                    entitled METHODS AND APPARATUS FOR DATA SMOOTHING;                -   claims the benefit of U.S. Provisional Patent                    Application No. 60/295,188, filed May 31, 2001,                    entitled METHODS AND APPARATUS FOR TEST DATA CONTROL                    AND ANALYSIS;                -   claims the benefit of U.S. Provisional Patent                    Application No. 60/374,328, filed Apr. 21, 2002,                    entitled METHODS AND APPARATUS FOR TEST PROGRAM                    ANALYSIS AND ENHANCEMENT, and                -   is a continuation-in-part of U.S. patent application                    Ser. No. 09/872,195, filed on May 31, 2001, now                    abandoned, entitled METHODS AND APPARATUS FOR DATA                    SMOOTHING and incorporates the disclosure of each                    application by reference;    -   a continuation-in-part of U.S. patent application Ser. No.        11/692,021, filed Mar. 27, 2007 now U.S. Pat. No. 8,041,541,        entitled METHODS AND APPARATUS FOR DATA ANALYSIS, which is a        continuation of U.S. patent application Ser. No. 10/730,388,        filed on Dec. 7, 2003, now U.S. Pat. No. 7,225,107 entitled        METHODS AND APPARATUS FOR DATA ANALYSIS, which:        -   claims the benefit of Provisional Patent Application No.            60/483,003, filed Jun. 27, 2003, entitled DEVICE INDEPENDENT            WAFERMAP ANALYSIS; and        -   is a continuation-in-part of U.S. patent application Ser.            No. 10/367,355, filed on Feb. 14, 2003, now U.S. Pat. No.            7,167,811 entitled METHODS AND APPARATUS FOR DATA ANALYSIS,            which is a continuation-in-part of U.S. patent application            Ser. No. 10/154,627, filed on May 24, 2002, now U.S. Pat.            No. 6,792,373 entitled METHODS AND APPARATUS FOR            SEMICONDUCTOR TESTING, which:            -   claims the benefit of U.S. Provisional Patent                Application No. 60/293,577, filed May 24, 2001, entitled                METHODS AND APPARATUS FOR DATA SMOOTHING;            -   claims the benefit of U.S. Provisional Patent                Application No. 60/295,188, filed May 31, 2001, entitled                METHODS AND APPARATUS FOR TEST DATA CONTROL AND                ANALYSIS;            -   claims the benefit of U.S. Provisional Patent                Application No. 60/374,328, filed Apr. 21, 2002,                entitled METHODS AND APPARATUS FOR TEST PROGRAM ANALYSIS                AND ENHANCEMENT; and            -   is a continuation-in-part of U.S. patent application                Ser. No. 09/872,195, filed on May 31, 2001, now                abandoned, entitled METHODS AND APPARATUS FOR DATA                SMOOTHING, and incorporates the disclosure of each                application by reference;    -   this application claims the benefit of U.S. Provisional Patent        Application No. 61/430,050, filed Jan. 5, 2001, entitled METHODS        AND APPARATUS FOR DATA ANALYSIS and incorporates the application        by reference. To the extent that the present disclosure        conflicts with any referenced application, however, the present        disclosure is to be given priority.

BACKGROUND OF THE INVENTION

Semiconductor companies test components to ensure that the componentsoperate properly. The test data not only determine whether thecomponents function properly, but also may indicate deficiencies in themanufacturing process. Accordingly, many semiconductor companies analyzethe collected data from several different components to identify andcorrect problems. For example, the company may gather test data formultiple chips on each wafer among several different lots. Test data maycome from a variety of sources, such as parametric electrical testing,optical inspection, scanning electron microscopy, energy dispersivex-ray spectroscopy, and focused ion beam processes for defect analysisand fault isolation. This data may be analyzed to identify commondeficiencies or patterns of defects or identify parts that may exhibitquality and performance issues and to identify or classify user-defined“good parts”. Steps may then be taken to correct the problems. Testingis typically performed before device packaging (at wafer level) as wellas upon completion of assembly (final test).

Gathering and analyzing test data is expensive and time consuming.Automatic testers apply signals to the components and read thecorresponding output signals. The output signals may be analyzed todetermine whether the component is operating properly. Each testergenerates a large volume of data. For example, each tester may perform200 tests on a single component, and each of those tests may be repeated10 times. Consequently, a test of a single component may yield 2000results. Because each tester is testing 100 or more components an hourand several testers may be connected to the same server, an enormousamount of data must be stored. Further, to process the data, the servertypically stores the test data in a database to facilitate themanipulation and analysis of the data. Storage in a conventionaldatabase, however, requires further storage capacity as well as time toorganize and store the data.

Furthermore, acquiring the test data presents a complex and painstakingprocess. A test engineer prepares a test program to instruct the testerto generate the input signals to the component and receive the outputsignals. The program tends to be very complex to ensure full and properoperation of the component. Consequently, the test program for amoderately complex integrated circuit involves a large number of testsand results. Preparing the program demands extensive design andmodification to arrive at a satisfactory solution, and optimization ofthe program, for example to remove redundant tests or otherwise minimizetest time, requires additional exertion.

The analysis of the gathered data is also difficult. The volume of thedata may demand significant processing power and time. As a result, thedata is not usually analyzed at product run time, but is insteadtypically analyzed between test runs or in other batches. To alleviatesome of these burdens, some companies only sample the data from thetesters and discard the rest. Analyzing less than all of the data,however, ensures that the resulting analysis cannot be fully completeand accurate. As a result, sampling degrades the complete understandingof the test results.

In addition, even when the full set of test data generated by the testeris retained, the sheer volume of the test data presents difficulties inanalyzing the data and extracting meaningful results. The data maycontain significant information about the devices, the testing process,and the manufacturing process that may be used to improve production,reliability, and testing. In view of the amount of data, however,isolating and presenting the information to the user or another systemis challenging.

Furthermore, much of the data interpretation is performed manually byengineers who review the data and make deductions about the test andmanufacturing process based on their experience and familiarity with thefabrication and test process. Although manual analysis is ofteneffective, engineers understand the fabrication and test systemsdifferently, and are thus prone to arriving at different subjectiveconclusions based on the same data. Another problem arises whenexperienced personnel leave the company or are otherwise unavailable,for their knowledge and understanding of the fabrication and test systemand the interpretation of the test data cannot be easily transferred toother personnel.

SUMMARY OF THE INVENTION

A method and apparatus for data analysis according to various aspects ofthe present invention is configured to test a set of components andgenerate test data for the components. A diagnostic system automaticallyanalyzes the test data to identify a characteristic of a componentfabrication process by recognizing a pattern in the test data andclassifying the pattern using a neural network.

BRIEF DESCRIPTION OF THE DRAWING

A more complete understanding of the present invention may be derived byreferring to the detailed description and the claims when considered inconnection with the following illustrative figures, which may not be toscale. Like reference numbers refer to similar elements throughout thefigures.

FIG. 1 is a block diagram of a test system according to various aspectsof the present invention and associated functional components;

FIG. 2 is a block diagram of elements for operating the test system;

FIG. 3 illustrates a flow diagram for a configuration element;

FIGS. 4A-C illustrate a flow diagram for a supplemental data analysiselement;

FIG. 5 is a diagram of various sections of a wafer and sectioningtechniques;

FIGS. 6A-B further illustrate a flow diagram for a supplemental dataanalysis element;

FIG. 7 illustrates a flow diagram for an output element;

FIG. 8 is a flow diagram for operation of an exemplary data smoothingsystem according to various aspects of the present invention;

FIG. 9 is a plot of test data for a test of multiple components;

FIG. 10 is a representation of a wafer having multiple devices and aresistivity profile for the wafer;

FIG. 11 is a graph of resistance values for a population of resistors inthe various devices of the wafer of FIG. 10 ;

FIGS. 12A-B are general and detailed plots, respectively, of raw testdata and outlier detection triggers for the various devices of FIG. 10 ;

FIG. 13 is a flow diagram of a composite analysis process according tovarious aspects of the present invention;

FIG. 14 is a diagram of a representative data point location on threerepresentative wafers;

FIGS. 15A-C are a flow diagram and a chart relating to a cumulativesquared composite data analysis process;

FIG. 16 is a diagram of an exclusion zone defined on a wafer;

FIGS. 17A-B are a flow diagram of a proximity weighting process;

FIG. 18 is a diagram of a set of data points subjects to proximityweighting;

FIG. 19 is a flow diagram of a cluster detection and filtration process;

FIG. 20 is a diagram of a set of clusters on a wafer subject todetection and filtration;

FIG. 21 is a diagram of a set of data points merged using an absolutemerge process;

FIG. 22 is a diagram of a set of data points merged using an overlapmerge process;

FIGS. 23 and 24 are diagrams of sets of data points merged usingpercentage overlap merge processes;

FIG. 25 is a block diagram of a system for identifying a characteristicof a process using test data;

FIG. 26 is a block diagram of a diagnostic system;

FIG. 27 is a flow diagram of a classification process;

FIG. 28 is a diagram of a pattern filtering process;

FIG. 29 is a diagram of a neural network;

FIG. 30 is a diagram of a system for automatically selecting one or moreoutlier identification algorithms;

FIGS. 31A-B are charts showing relationships between characteristics ofdifferent types of input data and possible causes of thecharacteristics;

FIG. 32 is a diagram of a classification process; and

Elements in the figures are illustrated for simplicity and clarity andhave not necessarily been drawn to scale. For example, the connectionsand steps performed by some of the elements in the figures may beexaggerated or omitted relative to other elements to help to improveunderstanding of embodiments of the present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

The present invention may be described in terms of functional blockcomponents and various process steps. Such functional blocks and stepsmay be realized by any number of hardware or software componentsconfigured to perform the specified functions. For example, the presentinvention may employ various testers, processors, storage systems,processes, and algorithms, e.g., statistical engines, memory elements,signal processing elements, neural networks, pattern analyzers, logicelements, programs, and the like, which may carry out a variety offunctions under the control of one or more testers, microprocessors, orother control devices. In addition, the present invention may bepracticed in conjunction with any number of test environments, and eachsystem described is merely one exemplary application for the invention.Further, the present invention may employ any number of conventionaltechniques for data analysis, component interfacing, data processing,component handling, and the like.

Referring to FIG. 1 , a method and apparatus according to variousaspects of the present invention operates in conjunction with a testsystem 100 having a tester 102, such as automatic test equipment (ATE)for testing semiconductors. In the present embodiment, the test system100 comprises a tester 102 and a computer system 108. The test system100 may be configured for testing any components 106, such assemiconductor devices on a wafer, circuit boards, packaged devices, orother electrical or optical systems. In the present embodiment, thecomponents 106 comprise multiple integrated circuit dies formed on awafer or packaged integrated circuits or devices. The components 106 arecreated using a fabrication process, which may comprise any suitablemanufacturing process for creating the components 106, and may include atest process, which may comprise any suitable process for testing theoperation of the components 106.

The tester 102 suitably comprises any test equipment that testscomponents 106 and generates output data relating to the testing, andmay comprise multiple machines or other sources of data. The tester 102may comprise a conventional automatic tester, such as a Teradyne tester,and suitably operates in conjunction with other equipment forfacilitating the testing. The tester 102 may be selected and configuredaccording to the particular components 106 to be tested and/or any otherappropriate criteria.

The tester 102 may operate in conjunction with the computer system 108to, for example, program the tester 102, load and/or execute the testprogram, collect data, provide instructions to the tester 102, analyzetest data, control tester parameters, and the like. In the presentembodiment, the computer system 108 receives tester data from the tester102 and performs various data analysis functions independently of thetester 102. The computer system 108 may implement a statistical engineto analyze data from the tester 102, as well as a diagnostic system 216for identifying potential problems in the fabrication and/or testprocess based on the test data. The computer system 108 may comprise aseparate computer, such as a personal computer or workstation, connectedto or networked with the tester 102 to exchange signals with the tester102. In an alternative embodiment, the computer system 108 may beomitted from or integrated into other components of the test system 100,and various functions may be performed by other components, such as thetester 102 or elements connected to the network.

In the present exemplary system, the computer system 108 includes aprocessor 110 and a memory 112. The processor 110 comprises any suitableprocessor, such as a conventional Intel, Motorola, or Advanced MicroDevices processor, operating in conjunction with any suitable operatingsystem, such as Windows XP, Unix, or Linux. Similarly, the memory 112may comprise any appropriate memory accessible to the processor 110,such as a random access memory (RAM) or other suitable storage system,for storing data. In particular, the memory 112 of the present systemincludes a fast access memory for storing and receiving information andis suitably configured with sufficient capacity to facilitate theoperation of the computer 108.

In the present embodiment, the memory 112 includes capacity for storingoutput results received from the tester 102 and facilitating analysis ofthe output test data. The memory 112 is configured for fast storage andretrieval of test data for analysis. In various embodiments, the memory112 is configured to store the elements of a dynamic datalog, suitablycomprising a set of information selected by the test system 100 and/orthe operator according to selected criteria and analyses based on thetest results.

For example, the memory 112 suitably stores a component identifier foreach component 106, such as x-y coordinates corresponding to a positionof the component 106 on a wafer map for the tested wafer. Each x-ycoordinate in the memory 112 may be associated with a particularcomponent 106 at the corresponding x-y coordinate on the wafer map. Eachcomponent identifier has one or more fields, and each field corresponds,for example, to a particular test performed on the component 106 at thecorresponding x-y position on the wafer, a statistic related to thecorresponding component 106, or other relevant data. The memory 112 maybe configured to include any data identified by the user as desiredaccording to any criteria or rules.

The computer 108 of the present embodiment also suitably has access to astorage system, such as another memory (or a portion of the memory 112),a hard drive array, an optical storage system, or other suitable storagesystem. The storage system may be local, like a hard drive dedicated tothe computer 108 or the tester 102, or may be remote, such as a harddrive array associated with a server to which the test system 100 isconnected. The storage system may store programs and/or data used by thecomputer 108 or other components of the test system 100. In the presentembodiment, the storage system comprises a database 114 available via aremote server 116 comprising, for example, a main production server fora manufacturing facility. The database 114 stores tester information,such as tester data files, master data files for operating the testsystem 100 and its components, test programs, downloadable instructionsfor the test system 100, and the like. In addition, the storage systemmay comprise complete tester data files, such as historical tester datafiles retained for analysis.

The test system 100 may include additional equipment to facilitatetesting of the components 106. For example, the present test system 100includes a device interface 104, like a conventional device interfaceboard and/or a device handler or prober, to handle the components 106and provide an interface between the components 106 and the tester 102.In one embodiment, the device interface comprises a multisite deviceinterface configured to simultaneously test multiple sites on a singlewafer. The test system 100 may include or be connected to othercomponents, equipment, software, and the like to facilitate testing ofthe components 106 according to the particular configuration,application, environment of the test system 100, or other relevantfactors. For example, in the present embodiment, the test system 100 isconnected to an appropriate communication medium, such as a local areanetwork, intranet, or global network like the internet, to transmitinformation to other systems, such as the remote server 116.

The test system 100 may include one or more testers 102 and one or morecomputers 108. For example, one computer 108 may be connected to anappropriate number of, such as up to twenty or more, testers 102according to various factors, such as the system's throughput and theconfiguration of the computer 108. Further, the computer 108 may beseparate from the tester 102, or may be integrated into the tester 102,for example utilizing one or more processors, memories, clock circuits,and the like of the tester 102 itself. In addition, various functionsmay be performed by different computers. For example, a first computermay perform various pre-analysis tasks, several computers may thenreceive the data and perform data analysis, and another set of computersmay prepare the dynamic datalogs and/or other output analyses andreports.

A test system 100 according to various aspects of the present inventiontests the components 106 and provides enhanced analysis and testresults. For example, the enhanced analysis may identify incorrect,questionable, or unusual results, repetitive tests, and/or tests with arelatively high probability of failure. The test system 100 may alsoanalyze multiple sets of data, such as data taken from multiple wafersand/or lots of wafers, to generate composite data based on multipledatasets. Various data may also be used by the test system 100 todiagnose characteristics in the fabrication, test, and/or other process,such as problems, inefficiencies, potential hazards, instabilities, orother aspects that may be identified via the test data. The operator,such as the product engineer, test engineer, manufacturing engineer,device engineer, or other personnel using the test data and analyses,may then use the results to verify and/or improve the test system 100and/or the fabrication system and classify the components 106.

The test system 100 according to various aspects of the presentinvention executes an enhanced test process for testing the components106 and collecting and analyzing test data. The test system 100 suitablyoperates in conjunction with a software application executed by thecomputer 108. Referring to FIG. 2 , the software application of thepresent embodiment includes multiple elements for implementing theenhanced test process, including a configuration element 202, asupplementary data analysis element 206, and an output element 208. Thetest system 100 may also include a composite analysis element 214 foranalyzing data from more than one dataset. Further, the test system mayinclude a diagnostic system 216 for identifying characteristics andpotential problems using the test data.

Each element 202, 206, 208, 214, 216 suitably comprises a softwaremodule operating on the computer 108 to perform various tasks.Generally, the configuration element 202 prepares test system 100 fortesting and analysis. In the supplementary data analysis element 206,output test data from the tester 102 is analyzed to generatesupplementary test data, suitably at run time and automatically. Thesupplementary test data is then transmitted to the operator or anothersystem, such as the composite analysis element 214, the diagnosticsystem 216, and/or the output element 208.

The configuration element 202 configures the test system 100 for testingthe components 106 and analyzing the test data. The test system 100suitably uses a predetermined set of initial parameters and, if desired,information from the operator to configure the test system 100. The testsystem 100 is suitably initially configured with predetermined ordefault parameters to minimize operator attendance to the test system100. Adjustments may be made to the configuration by the operator, ifdesired, for example via the computer 108.

Referring to FIG. 3 , an exemplary configuration process 300 performedby the configuration element 202 begins with an initialization procedure(step 302) to set the computer 108 in an initial state. Theconfiguration element 202 then obtains application configurationinformation (step 304), for example from the database 114, for thecomputer 108 and the tester 102. For example, the configuration element202 may access a master configuration file for the enhanced test processand/or a tool configuration file relating to the tester 102. The masterconfiguration file may contain data relating to the proper configurationfor the computer 108 and other components of the test system 100 toexecute the enhanced test process. Similarly, the tool configurationfile suitably includes data relating to the tester 102 configuration,such as connection, directory, IP address, tester node identification,manufacturer, flags, prober identification, or any other pertinentinformation for the tester 102.

The configuration element 202 may then configure the test system 100according to the data contained in the master configuration file and/orthe tool configuration file (step 306). In addition, the configurationelement 202 may use the configuration data to retrieve further relevantinformation from the database 114, such as the tester's 102 identifier(step 308) for associating data like logistics instances for tester datawith the tester 102. The test system 100 information also suitablyincludes one or more default parameters that may be accepted, declined,or adjusted by the operator. For example, the test system 100information may include global statistical process control (SPC) rulesand goals that are submitted to the operator upon installation,configuration, power-up, or other appropriate time for approval and/ormodification. The test system 100 information may also include defaultwafer maps or other files that are suitably configured for each product,wafer, component 106, or other item that may affect or be affected bythe test system 100. The configuration algorithms, parameters, and anyother criteria may be stored in a recipe file for easy access,correlation to specific products and/or tests, and for traceability.

When the initial configuration process is complete, the test system 100commences a test run, for example in conjunction with a conventionalseries of tests, in accordance with a test program. The tester 102suitably executes the test program to apply signals to connections onthe components 106 and read output test data from the components 106.The tester 102 may perform multiple tests on each component 106 on awafer or the wafer itself, and each test may be repeated several timeson the same component 106. The tests may comprise any appropriate tests,such as (but not limited to) continuity, supply current, leakagecurrent, parametric static, parametric dynamic, and functional andstress tests. Test data from the tester 102 is stored for quick accessand supplemental analysis as the test data is acquired. The data mayalso be stored in a long-term memory for subsequent analysis and use.

Each test generates at least one result for at least one of thecomponents. Referring to FIG. 9 , an exemplary set of test results for asingle test of multiple components comprises a first set of test resultshaving statistically similar values and a second set of test resultscharacterized by values that stray from the first set. Each test resultmay be compared to an upper test limit and a lower test limit. If aparticular result for a component exceeds either limit, the componentmay be classified as a “bad part” or otherwise classified according tothe test and/or the test result.

Some of the test results in the second set that stray from the first setmay exceed the control limits, while others do not. For the presentpurposes, those test results that stray from the first set but do notexceed the control limits or otherwise fail to be detected are referredto as “outliers”. The outliers in the test results may be identified andanalyzed for any appropriate purpose, such as to identify potentiallyunreliable components. The outliers may also be used to identify avarious potential problems and/or improvements in the test andmanufacturing processes.

As the tester 102 generates the test results, the output test data foreach component, test, and repetition is stored by the tester 102 in atester data file. The output test data received from each component 106is analyzed by the tester 102 to classify the performance of thecomponent 106, such as into a particular bin classification, for exampleby comparison to the upper and lower test limits, and the results of theclassification are also stored in the tester data file. The tester datafile may include additional information as well, such as logistics dataand test program identification data. The tester data file is thenprovided to the computer 108 in an output file, such as a standardtester data format (STDF) file, and stored in memory. The tester datafile may also be stored in the storage system for longer term storagefor later analysis, such as by the composite analysis element 214.

When the computer 108 receives the tester data file, the supplementarydata analysis element 206 analyzes the data to provide enhanced outputresults. The supplementary data analysis element 206 may provide anyappropriate analysis of the tester data to achieve any suitableobjective. For example, the supplementary data analysis element 206 mayimplement a statistical engine for analyzing the output test data at runtime and identifying data and characteristics of the data of interest tothe operator. The data and characteristics identified may be stored,while data that is not identified may be otherwise disposed of, such asdiscarded.

The supplementary data analysis element 206 may, for example, calculatestatistical figures according to the data and a set of statisticalconfiguration data. The statistical configuration data may call for anysuitable type of analysis according to the needs of the test system 100and/or the operator, such as statistical process control, outlieridentification and classification, signature analyses, and datacorrelation. Further, the supplementary data analysis element 206suitably performs the analysis at run time, i.e. within a matter ofseconds or minutes following generation of the test data. Thesupplementary data analysis element 206 may also perform the analysisautomatically with minimal intervention from the operator and/or testengineer.

In the present test system 100, after the computer 108 receives andstores the tester data file, the supplementary data analysis element 206performs various preliminary tasks to prepare the computer 108 foranalysis of the output test data and facilitate generation ofsupplementary data and preparation of an output report. Referring now toFIGS. 4A-C, in the present embodiment, the supplementary data analysiselement 206 initially copies the tester data file to a tool inputdirectory corresponding to the relevant tester 102 (step 402). Thesupplementary data analysis element 206 also retrieves configurationdata to prepare the computer 108 for supplementary analysis of theoutput test data.

The configuration data suitably includes a set of logistics data thatmay be retrieved from the tester data file (step 404). The supplementarydata analysis element 206 also creates a logistics reference (step 406).The logistics reference may include tester 102 information, such as thetester 102 information derived from the tool configuration file. Inaddition, the logistics reference is assigned an identification.

The configuration data may also include an identifier for the testprogram that generated the output test data. The test program may beidentified in any suitable manner, such as looking it up in the database114 (step 408), by association with the tester 102 identification, orreading it from the master configuration file. If no test programidentification can be established (step 410), a test programidentification may be created and associated with the testeridentification (step 412).

The configuration data further identifies the wafers in the test run tobe processed by the supplementary data analysis element 206, if fewerthan all of the wafers. In the present embodiment, the supplementarydata analysis element 206 accesses a file indicating which wafers are tobe analyzed (step 414). If no indication is provided, the computer 108suitably defaults to analyzing all of the wafers in the test run.

If the wafer for the current test data file is to be analyzed (step416), the supplementary data analysis element 206 proceeds withperforming the supplementary data analysis on the test data file for thewafer. Otherwise, the supplementary data analysis element 206 waits foror accesses the next test data file (step 418).

The supplementary data analysis element 206 may establish one or moresection groups to be analyzed for the various wafers to be tested (step420). To identify the appropriate section group to apply to the outputtest data, the supplementary data analysis element 206 suitablyidentifies an appropriate section group definition, for exampleaccording to the test program and/or the tester identification. Eachsection group includes one or more section arrays, and each sectionarray includes one or more sections of the same section types.

Section types comprise various sorts of component 106 groups positionedin predetermined areas of the wafer. For example, referring to FIG. 5 ,a section type may include a row 502, a column 504, a stepper field 506,a circular band 508, a radial zone 510, a quadrant 512, or any otherdesired grouping of components. Different section types may be usedaccording to the configuration of the components, such as order ofcomponents processed, sections of a tube, or the like. Such groups ofcomponents 106 are analyzed together to identify, for example, commondefects or characteristics that may be associated with the group. Forexample, if a particular portion of the wafer does not conduct heat likeother portions of the wafer, the test data for a particular group ofcomponents 106 may reflect common characteristics or defects associatedwith the uneven heating of the wafer.

Upon identifying the section group for the current tester data file, thesupplemental data analysis element 206 retrieves any further relevantconfiguration data, such as control limits and enable flags for the testprogram and/or tester 102 (step 422). In particular, the supplementaldata analysis element 206 suitably retrieves a set of desired statisticsor calculations associated with each section array in the section group(step 423). Desired statistics and calculations may be designated in anymanner, such as by the operator or retrieved from a file. Further, thesupplemental data analysis element 206 may also identify one or moresignature analysis algorithms (step 424) for each relevant section typeor other appropriate variation relating to the wafer and retrieve thesignature algorithms from the database 114 as well.

All of the configuration data may be provided by default orautomatically accessed by the configuration element 202 or thesupplemental data analysis element 206. Further, the configurationelement 202 and the supplemental data analysis element 206 of thepresent embodiment suitably allow the operator to change theconfiguration data according to the operator's wishes or the test system100 requirements. When the configuration data have been selected, theconfiguration data may be associated with relevant criteria and storedfor future use as default configuration data. For example, if theoperator selects a certain section group for a particular kind ofcomponents 106, the computer 108 may automatically use the same sectiongroup for all such components 106 unless instructed otherwise by theoperator.

The supplemental data analysis element 206 also provides forconfiguration and storage of the tester data file and additional data.The supplemental data analysis element 206 suitably allocates memory(step 426), such as a portion of the memory 112, for the data to bestored. The allocation suitably provides memory for all of the data tobe stored by the supplemental data analysis element 206, includingoutput test data from the tester data file, statistical data generatedby the supplemental data analysis element 206, control parameters, andthe like. The amount of memory allocated may be calculated according to,for example, the number of tests performed on the components 106, thenumber of section group arrays, the control limits, statisticalcalculations to be performed by the supplementary data analysis element206, and the like.

When all of the configuration data for performing the supplementaryanalysis are ready and upon receipt of the output test data, thesupplementary data analysis element 206 loads the relevant test datainto memory (step 428) and performs the supplementary analysis on theoutput test data. The supplementary data analysis element 206 mayperform any number and types of data analyses according to thecomponents 106, configuration of the test system 100, desires of theoperator, or other relevant criteria. The supplemental data analysiselement 206 may be configured to analyze the sections for selectedcharacteristics identifying potentially defective components 106 andpatterns, trends, or other characteristics in the output test data thatmay indicate manufacturing concerns or flaws.

The present supplementary data analysis element 206, for example,smoothes the output test data, calculates and analyzes variousstatistics based on the output test data, and identifies data and/orcomponents 106 corresponding to various criteria. The presentsupplementary data analysis element 206 may also classify and correlatethe output test data to provide information to the operator and/or testengineer relating to the components 106 and the test system 100. Forexample, the present supplementary data analysis element 206 may performoutput data correlations, for example to identify potentially related orredundant tests, and an outlier incidence analysis to identify testshaving frequent outliers.

The supplementary data analysis element 206 may include a smoothingsystem to initially process the tester data to smooth the data andassist in the identification of outliers (step 429). In alternativeembodiments, the smoothing system and process may be omitted and thedata processed without smoothing. The smoothing system may also identifysignificant changes in the data, trends, and the like, which may beprovided to the operator by the output element 208. The smoothing systemis suitably implemented, for example, as a program operating on thecomputer system 108. The smoothing system suitably comprises multiplephases for smoothing the data according to various criteria. The firstphase may include a basic smoothing process. The supplemental phasesconditionally provide for enhanced tracking and/or additional smoothingof the test data.

The smoothing system suitably operates by initially adjusting an initialvalue of a selected tester datum according to a first smoothingtechnique, and supplementally adjusting the value according to a secondsmoothing technique if at least one of the initial value and theinitially adjusted value meets a threshold. The first smoothingtechnique tends to smooth the data. The second smoothing technique alsotends to smooth the data and/or improve tracking of the data, but in adifferent manner from the first smoothing technique. Further, thethreshold may comprise any suitable criteria for determining whether toapply supplemental smoothing. The smoothing system suitably compares aplurality of preceding adjusted data to a plurality of preceding rawdata to generate a comparison result, and applies a second smoothingtechnique to the selected datum to adjust the value of the selecteddatum according to whether the comparison result meets a firstthreshold. Further, the smoothing system suitably calculates a predictedvalue of the selected datum, and may apply a third smoothing techniqueto the selected datum to adjust the value of the selected datumaccording to whether the predicted value meets a second threshold.

Referring to FIG. 8 , a first smoothed test data point is suitably setequal to a first raw test data point (step 802) and the smoothing systemproceeds to the next raw test data point (step 804). Before performingsmoothing operations, the smoothing system initially determines whethersmoothing is appropriate for the data point and, if so, performs a basicsmoothing operation on the data. Any criteria may be applied todetermine whether smoothing is appropriate, such as according to thenumber of data points received, the deviation of the data point valuesfrom a selected value, or comparison of each data point value to athreshold. In the present embodiment, the smoothing system performs athreshold comparison. The threshold comparison determines whether datasmoothing is appropriate. If so, the initial smoothing process issuitably configured to proceed to an initial smoothing of the data.

More particularly, in the present embodiment, the process starts with aninitial raw data point R₀, which is also designated as the firstsmoothed data point S₀. As additional data points are received andanalyzed, a difference between each raw data point (R_(n)) and apreceding smoothed data point (S_(n-1)) is calculated and compared to athreshold (T₁) (step 806). If the difference between the raw data pointR_(n) and the preceding smoothed data point S_(n-1) exceeds thethreshold T₁, it is assumed that the exceeded threshold corresponds to asignificant departure from the smoothed data and indicates a shift inthe data. Accordingly, the occurrence of the threshold crossing may benoted and the current smoothed data point S_(n) is set equal to the rawdata point R_(n) (step 808). No smoothing is performed, and the processproceeds to the next raw data point.

If the difference between the raw data point and the preceding smootheddata point does not exceed the threshold T₁, the process calculates acurrent smoothed data point S_(n) in conjunction with an initialsmoothing process (step 810). The initial smoothing process provides abasic smoothing of the data. For example, in the present embodiment, thebasic smoothing process comprises a conventional exponential smoothingprocess, such as according to the following equation:S _(n)=(R _(n) −S _(n-1))*M ₁ +S _(n-1)

where M₁ is a selected smoothing coefficient, such as 0.2 or 0.3.

The initial smoothing process suitably uses a relatively low coefficientM₁ to provide a significant amount of smoothing for the data. Theinitial smoothing process and coefficients may be selected according toany criteria and configured in any manner, however, according to theapplication of the smoothing system, the data processed, requirementsand capabilities of the smoothing system, and/or any other criteria. Forexample, the initial smoothing process may employ random, random walk,moving average, simple exponential, linear exponential, seasonalexponential, exponential weighted moving average, or any otherappropriate type of smoothing to initially smooth the data.

The data may be further analyzed for and/or subjected to smoothing.Supplementary smoothing may be performed on the data to enhance thesmoothing of the data and/or improve the tracking of the smoothed datato the raw data. Multiple phases of supplementary smoothing may also beconsidered and, if appropriate, applied. The various phases may beindependent, interdependent, or complementary. In addition, the data maybe analyzed to determine whether supplementary smoothing is appropriate.

In the present embodiment, the data is analyzed to determine whether toperform one or more additional phases of smoothing. The data is analyzedaccording to any appropriate criteria to determine whether supplementalsmoothing may be applied (step 812). For example, the smoothing systemidentify trends in the data, such as by comparing a plurality ofadjusted data points and raw data points for preceding data andgenerating a comparison result according to whether substantially all ofthe preceding adjusted data share a common relationship (such as lessthan, greater than, or equal to) with substantially all of thecorresponding raw data.

The smoothing system of the present embodiment compares a selectednumber P₂ of raw data points to an equal number of smoothed data points.If the values of all of the P₂ raw data points exceed (or are equal to)the corresponding smoothed data points, or if all raw data points areless than (or equal to) the corresponding smoothed data points, then thesmoothing system may determine that the data is exhibiting a trend andshould be tracked more closely. Accordingly, the occurrence may be notedand the smoothing applied to the data may be changed by applyingsupplementary smoothing. If, on the other hand, neither of thesecriteria is satisfied, then the current smoothed data point remains asoriginally calculated and the relevant supplementary data smoothing isnot applied.

In the present embodiment, the criterion for comparing the smoothed datato the raw data is selected to identify a trend in the data behind whichthe smoothed data may be lagging. Accordingly, the number of points P₂may be selected according to the desired sensitivity of the system tochanging trends in the raw data.

The supplementary smoothing changes the effect of the overall smoothingaccording to the data analysis. Any appropriate supplementary smoothingmay be applied to the data to more effectively smooth the data or tracka trend in the data. For example, in the present embodiment, if the dataanalysis indicates a trend in the data that should be tracked moreclosely, then the supplementary smoothing may be applied to reduce thedegree of smoothing initially applied so that the smoothed data moreclosely tracks the raw data (step 814).

In the present embodiment, the degree of smoothing is reduced byrecalculating the value for the current smoothed data point using areduced degree of smoothing. Any suitable smoothing system may be usedto more effectively track the data or otherwise respond to the resultsof the data analysis. In the present embodiment, another conventionalexponential smoothing process is applied to the data using a highercoefficient M₂:S _(n)=(R _(n) −S _(n-1))*M ₂ +S _(n-1)

The coefficients M₁ and M₂ may be selected according to the desiredsensitivity of the system, both in the absence (M₁) and the presence(M₂) of trends in the raw data. In various applications, for example,the value of M₁ may be higher than the value of M₂.

The supplementary data smoothing may include additional phases as well.The additional phases of data smoothing may similarly analyze the datain some manner to determine whether additional data smoothing should beapplied. Any number of phases and types of data smoothing may be appliedor considered according to the data analysis.

For example, in the present embodiment, the data may be analyzed andpotentially smoothed for noise control, such as using a predictiveprocess based on the slope, or trend, of the smoothed data. Thesmoothing system computes a slope (step 816) based on a selected numberP₃ of smoothed data points preceding the current data point according toany appropriate process, such as linear regression, N-points centered,or the like. In the present embodiment, the data smoothing system uses a“least squares fit through line” process to establish a slope of thepreceding P₃ smoothed data points.

The smoothing system predicts a value of the current smoothed data pointaccording to the calculated slope. The system then compares thedifference between the previously calculated value for the currentsmoothed data point (S_(n)) to the predicted value for the currentsmoothed data point to a range number (R₃) (step 818). If the differenceis greater than the range R₃, then the occurrence may be noted and thecurrent smoothed data point is not adjusted. If the difference is withinthe range R₃, then the current smoothed data point is set equal to thedifference between the calculated current smoothed data point (S_(n))and the predicted value for the current smoothed data point (S_(n-pred))multiplied by a third multiplier M₃ and added to the original value ofthe current smoothed data point (step 820). The equation:S _(n)=(S _(n-pred) −S _(n))*M ₃ +S _(n)

Thus, the current smoothed data point is set according to a modifieddifference between the original smoothed data point and the predictedsmoothed data point, but reduced by a certain amount (when M₃ is lessthan 1). Applying the predictive smoothing tends to reducepoint-to-point noise sensitivity during relatively flat (or otherwisenon-trending) portions of the signal. The limited application of thepredictive smoothing process to the smoothed data points ensures thatthe calculated average based on the slope does not affect the smootheddata when significant changes are occurring in the raw data, i.e., whenthe raw data signal is not relatively flat.

The supplementary data analysis element 206 may perform any appropriateanalysis of the tester data, including the ray tester data, the smoothedtester data, or otherwise filtered or processed data. For example, thesupplementary data analysis element 206 may filter the tester data toextract information about the nature of the distribution to choose theright technique to detect the failures, alarms, and tail thresholds. Thesupplementary data analysis element 206 may conduct statistical processcontrol (SPC) calculations and analyses on the output test data. Moreparticularly, referring again to FIG. 4A-C, the supplemental dataanalysis element 206 may calculate and store desired statistics for aparticular component, test, and/or section (step 430). The statisticsmay comprise any statistics useful to the operator or the test system100, such as SPC figures that may include averages, standard deviations,minima, maxima, sums, counts, Cp, Cpk, or any other appropriatestatistics.

The supplementary data analysis element 206 also suitably performs asignature analysis to dynamically and automatically identify trends andanomalies in the data, for example according to section, based on acombination of test results for that section and/or other data, such ashistorical data (step 442). The signature analysis identifies signaturesand applies a weighting system, suitably configured by the operator,based on any suitable data, such as the test data or identification ofdefects. The signature analysis may cumulatively identify trends andanomalies that may correspond to problem areas or other characteristicsof the wafer or the fabrication process. Signature analysis may beconducted for any desired signatures, such as noise peaks, waveformvariations, mode shifts, and noise. In the present embodiment, thecomputer 108 suitably performs the signature analysis on the output testdata for each desired test in each desired section.

In the present embodiment, a signature analysis process may be performedin conjunction with the smoothing process. As the smoothing processanalyzes the tester data, results of the analysis indicating a trend oranomaly in the data are stored as being indicative of a change in thedata or an outlier that may be of significance to the operator and/ortest engineer. For example, if a trend is indicated by a comparison ofsets of data in the smoothing process, the occurrence of the trend maybe noted and stored. Similarly, if a data point exceeds the threshold T₁in the data smoothing process, the occurrence may be noted and storedfor later analysis and/or inclusion in the output report. Alternatively,the smoothing process may be omitted.

For example, referring to FIGS. 6A-B, a signature analysis process 600may initially calculate a count (step 602) for a particular set of testdata and control limits corresponding to a particular section and test.The signature analysis process then applies an appropriate signatureanalysis algorithm to the data points (step 604). The signature analysisis performed for each desired signature algorithm, and then to each testand each section to be analyzed. Errors identified by the signatureanalysis, trend results, and signature results are also stored (step606). The process is repeated for each signature algorithm (step 608),test (step 610), and section (step 612). Upon completion, thesupplementary data analysis element 206 records the errors (step 614),trend results (step 616), signature results (step 618), and any otherdesired data in the storage system.

Upon identification of each relevant data point, such as outliers andother data of importance identified by the supplementary analysis, eachrelevant data point may be associated with a value identifying therelevant characteristics (step 444). For example, each relevantcomponent or data point may be associated with a series of values,suitably expressed as a hexadecimal figure, corresponding to the resultsof the supplementary analysis relating to the data point. Each value mayoperate as a flag or other designator of a particular characteristic.For example, if a particular data point has failed a particular testcompletely, a first flag in the corresponding hexadecimal value may beset. If a particular data point is the beginning of a trend in the data,another flag may be set. Another value in the hexadecimal figure mayinclude information relating to the trend, such as the duration of thetrend in the data.

The supplementary data analysis element 206 may also be configured toclassify and correlate the data (step 446). For example, thesupplementary data analysis element 206 may utilize the information inthe hexadecimal figures associated with the data points to identify thefailures, outliers, trends, and other features of the data. Thesupplementary data analysis element 206 also suitably appliesconventional correlation techniques to the data, for example to identifypotentially redundant or related tests.

The computer 108 may perform additional analysis functions upon thegenerated statistics and the output test data, such as automaticallyidentifying and classifying outliers (step 432). Analyzing each relevantdatum according to the selected algorithm suitably identifies theoutliers. If a particular algorithm is inappropriate for a set of data,the supplementary data analysis element 206 may be configured toautomatically abort the analysis and select a different algorithm.

The supplementary data analysis element 206 may operate in any suitablemanner to designate outliers, such as by comparison to selected valuesand/or according to treatment of the data in the data smoothing orfiltering process. For example, an outlier identification elementaccording to various aspects of the present invention initiallyautomatically calibrates its sensitivity to outliers based on selectedstatistical relationships for each relevant datum (step 434). Some ofthese statistical relationships are then compared to a threshold orother reference point, such as the data mode, mean, or median, orcombinations thereof, to define relative outlier threshold limits. Inthe present embodiment, the statistical relationships are scaled, forexample by one, two, three, and six standard deviations of the data, todefine the different outlier amplitudes (step 436). The output test datamay then be compared to the outlier threshold limits to identify andclassify the output test data as outliers (step 438).

The supplementary data analysis element 206 stores the resultingstatistics and outliers in memory and identifiers, such as the x-y wafermap coordinates, associated with any such statistics and outliers (step440). Selected statistics, outliers, and/or failures may also triggernotification events, such as sending an electronic message to anoperator, triggering a light tower, stopping the tester 102, ornotifying a server.

In the present embodiment, the supplementary data analysis element 206includes a scaling element 210 and an outlier classification element212. The scaling element 210 is configured to dynamically scale selectedcoefficients and other values according to the output test data. Theoutlier classification element 212 is configured to identify and/orclassify the various outliers in the data according to selectedalgorithms.

More particularly, the scaling element of the present embodimentsuitably uses various statistical relationships for dynamically scalingoutlier sensitivity and smoothing coefficients for noise filteringsensitivity. The scaling coefficients are suitably calculated by thescaling element and used to modify selected outlier sensitivity valuesand smoothing coefficients. Any appropriate criteria, such as suitablestatistical relationships, may be used for scaling. Alternatively,scaling may be omitted from the process.

For example, a sample statistical relationship for outlier sensitivityscaling is defined as:(√{square root over (1+NaturalLog_(Cpk) ²)})

Another sample statistical relationship for outlier sensitivity andsmoothing coefficient scaling is defined as:(√{square root over (1+NaturalLog_(Cpk) ²)})*Cpm

Another sample statistical relationship for outlier sensitivity andsmoothing coefficient scaling is defined as:

$\frac{\left( {\sigma*{Cpk}} \right)}{\left( {{Max} - {Min}} \right)},$where σ=datum Standard Deviation

A sample statistical relationship used in multiple algorithms forsmoothing coefficient scaling is:

${\frac{\sigma}{\mu}*10},$where σ=datum Standard Deviation and μ=datum Mean

Another sample statistical relationship used in multiple algorithms forsmoothing coefficient scaling is:

${\frac{\sigma^{2}}{\mu^{2}}*10},$where σ=datum Standard Deviation and μ=datum Mean

The outlier classification element 212 is suitably configured toidentify and/or classify the outliers in the components 106, output testdata, and/or analysis results according to any suitable algorithms. Inaddition, the outlier classification element 212 may be configured toutilize multiple candidate outlier identification algorithms andidentify one or more algorithms suited for identifying outliers in theoutput test data. Different tests generate different populationdistributions, such that an outlier identification algorithm that isappropriate for one test may be inappropriate for another. The outlierclassification element 212 is suitably configured to differentiatebetween different data populations and automatically select one or moreoutlier identification algorithms based on the data population type ofthe current data. The automatic selection may select from anyappropriate set of candidate outlier identification algorithms, and mayperform the selection according to any suitable criteria and analysis.

For example, referring to FIG. 30 , the outlier classification element212 may be configured to automatically perform an outlier identificationalgorithm selection process. The outlier classification element 212suitably comprises a pre-processing engine 3010 and a classificationengine 3012. The pre-processing engine 3010 suitably generates data tofacilitate selection of the relevant outlier identification algorithms.The classification engine 3012 suitably selects one or more relevantoutlier identification algorithms and identifies the outliersaccordingly.

The output test data, for example data taken from a particular test, areinitially provided to the outlier classification element 212 to analyzethe output test data for compatibility with various candidate outlieridentification algorithms. The data may be analyzed in any suitablemanner to identify appropriate algorithms for identifying the outliersin the output test data. For example, in the present embodiment, thepre-processing engine 3010 receives the output test data and preparesthe available outlier identification algorithms, such as by retrievingthem from an outlier identification algorithm library stored in memory.The pre-processing engine 3010 analyzes the output test data foroutliers using several of the available algorithms. In the presentembodiment, the pre-processing engine 3010 analyzes the output test datausing each of the algorithms designated by the user, or another suitableselection of algorithms, to generate pre-processing data, such asoutliers as identified by all algorithms and various descriptivestatistics, such as minimum, maximum, mean, median, standard deviation,CPK, CPM, and the like.

The algorithms may be based on industry standard (e.g., IQR,median+/−N*sigma, etc.) and/or proprietary, custom, or user-definedoutlier identification techniques. The outlier identification algorithmlibrary is suitably configurable by the user, for example to add,remove, or edit outlier identification algorithms, for example accordingto the particular products under test or the characteristics of thetests to be performed. Different algorithms may be appropriate fordifferent statistical population types, such as normal, logarithmicnormal, bimodal, clamped, or low CPK data populations. The candidateoutlier identification algorithms may comprise any suitable algorithmsfor various types and distributions of data, such as inter-quartilerange (IQR) normal distribution, 3 sigma; IQR normal distribution, 6sigma; IQR log normal, 3 sigma; IQR log normal, 6 sigma; bimodalalgorithms; clamped algorithms; low capability algorithms; customalgorithms based on 3-, 6-, or n-sigma; and proprietary algorithmshaving various sensitivities. The pre-processing engine 3010 may alsoanalyze the test data to generate characteristics relating to the outputtest data. For example, the pre-processing engine 3010 may calculatevarious statistical properties of the output test data.

The pre-processing algorithm results are dynamically selected foroutlier detection. In the present embodiment, the outlier classificationelement 212 analyzes the test results to identify the most useful orapplicable outlier identification algorithms. The data from the selectedoutlier identification algorithms may be retained, while the remainingdata is discarded. For example, in the present embodiment, theclassification engine 3012 receives the results of the pre-processinganalysis generated by each of the available outlier identificationalgorithms. The classification engine 3012 analyzes the pre-processingdata according to any suitable criteria, such as predetermined and/oruser-defined recipe-driven rules to determine whether the pre-processingdata satisfy various criteria.

The rules may be any appropriate rules, for example employingstatistical ratios or values, such as comparing statistics, likeminimum, maximum, mean, median, standard deviation, CPK, and CPM, tovarious thresholds or other criteria. For example, the classificationengine 3012 may skip the outlier detection process under certaincircumstances, such as having too few test results or a too narrow or abimodal distribution among the test results. The rules may bepre-selected and/or may be adjusted or added by the user to accommodatespecific conditions of the products and test environment. Further, theclassification engine 3012 may be configured to apply a particularalgorithm to a certain type of test, for example when the results of thetest are known to have a particular distribution. Other rules maydetermine whether a particular test is applicable. For example, theclassification engine 3012 may compare the CPK to a threshold. If theCPK is below the threshold, then the IQR normal outlier identificationalgorithm may be used. In the present system, results from an algorithmsatisfying a rule are used for outlier identification. Other algorithmresults for that test are suitably ignored.

The outlier classification element 212 may also identify and classifyselected outliers and components 106 according to the test output testresults and the information generated by the supplementary analysiselement 206. For example, the outlier classification element 212 issuitably configured to classify the components 106 intocritical/marginal/good part categories, for example in conjunction withuser-defined criteria; user-defined good/bad spatial patternsrecognition; classification of pertinent data for tester datacompression; test setup in-situ sensitivity qualifications and analysis;tester yield leveling analysis; dynamic wafer map and/or test stripmapping for part dispositions and dynamic retest; or test programoptimization analyses. The outlier classification element 212 mayclassify data in accordance with conventional SPC control rules, such asWestern Electric rules or Nelson rules to characterize the data.

The outlier classification element 212 suitably classifies the datausing a selected set of classification limit calculation methods. Anyappropriate classification methods may be used to characterize the dataaccording to the needs of the operator. The present outlierclassification element 212, for example, classifies outliers bycomparing the output test data to selected thresholds, such as valuescorresponding to one, two, three, and six statistically scaled standarddeviations from a threshold, such as the data mean, mode, and/or median.The identification of outliers in this manner tends to normalize anyidentified outliers for any test regardless of datum amplitude andrelative noise.

The outlier classification element 212 analyzes and correlates thenormalized outliers and/or the raw data points based on user-definedrules, which may comprise any suitable techniques for part and/orpattern classification. Sample user-selectable methods for the purposeof part and pattern classification based on identified outliers are asfollows:

Cumulative Amplitude, Cumulative Count Method:

${{Count}_{LIMIT} = {\mu_{OverallOutlierCount} + \left( \frac{3*\left( \sigma_{OverallOutlierCount}^{2} \right)}{\left( {{Max}_{OverallOutlierCount} - {Min}_{OverallOutlierCount}} \right)} \right)}}{{NormalizedOutlierAmplitude}_{LIMIT} = {\mu_{{Overall}{Normalized}{Outlier}{Amplitude}} + \left( \frac{3*\left( \sigma_{{Overall}{Normalized}{Outlier}{Amplitude}}^{2} \right)}{\begin{pmatrix}{{Max}_{{Overall}{Normalized}{Outlier}{Amplitude}} -} \\{Min}_{{Overall}{Normalized}{OutlierAmplitude}}\end{pmatrix}} \right)}}$

Classification Rules:Part_(CRITICAL)=True,If└(Part_(Cumlative Outlier Count)>Count_(LIMIT))AND(Part_(Cumlative Normalized OutlierAmplitude)>NormalizedOutlierAmplitude_(LIMIT))┘Part_(MARGINAL:HighAmplitude)=True,If[(Part_(Cumlative Normalized OutlierAmplitude)>NormalizedOutlierAmplitude_(LIMIT))]Part_(MARGINAL:HighCount)=True,If(Part_(CumlativeOutlierCount)>Count_(LIMIT))

Cumulative Amplitude Squared, Cumulative Count Squared Method:

${{Count}_{{LIMIT}^{2}} = {\mu_{{OverallOutlierCount}^{2}} + \left( \frac{3*\left( \sigma_{{OverallOutlierCount}^{2}}^{2} \right)}{\left( {{Max}_{{OverallOutlierCount}^{2}} - {Min}_{{OverallOutlierCount}^{2}}} \right)} \right)}}{{{Normalized}{Outlier}{Amplitude}_{{LIMIT}^{2}}} = {\mu_{{Overall}{Normalized}{Outlier}{Amplitude}^{2}} + \left( \frac{3*\left( \sigma_{{Overall}{Normalized}{Outlier}{Amplitude}^{2}}^{2} \right)}{\begin{pmatrix}{{Max}_{{Overall}{Normalized}{Outlier}{Amplitude}^{2}} -} \\{Min}_{{Overall}{Normalized}{Outlier}{Amplitude}^{2}}\end{pmatrix}} \right)}}$

Classification Rules:Part_(CRITICAL)=True,If└(Part_(CumlativeOutlierCount) ₂ >Count_(LIMIT) ₂)AND(Part_(CumlativeNormalizedOutlierAmpltude) ₂ >NormalizedOutlierAmplituide_(LIMIT) ₂ )┘Part_(MARGINAL:HighAmplitude)=True,If└(Part_(CumlativeNormalizedOutlierAmplitude)₂ >Normalized OutlierAmplitude_(LIMIT) ₂ )┘Part_(MARGINAL:HighCount)=True,If└(Part_(Cumlative Outlier Count) ₂>Count_(LIMIT) ₂ )┘

N-Points Method:

The actual numbers and logic rules used in the following examples can becustomized by the end user per scenario (test program, test node,tester, prober, handler, test setup, etc.). σ in these examples=σrelative to datum mean, mode, and/or median based on datum standarddeviation scaled by key statistical relationships.Part_(CRITICAL)=True,If[(Part_(COUNT:6σ)+Part_(COUNT:3σ))≥2)OR((Part_(COUNT:2σ)+Part_(COUNT:1σ))≥6)]Part_(CRITICAL)=True,If[(Part_(COUNT:6σ)+Part_(COUNT:3σ))≥1)AND((Part_(COUNT:2σ)+Part_(COUNT:1σ))≥3)]Part_(MARGINAL)=True,If[(Part_(COUNT:6σ)+Part_(COUNT:3σ)+Part_(COUNT:2σ)+Part_(COUNT:1σ))≥3)]Part_(NOISY)=True,If[((Part_(COUNT:6σ)+Part_(COUNT:3σ)+Part_(COUNT:2σ)+Part_(COUNT:1σ))≥1)]

The supplementary data analysis element 206 may be configured to performadditional analysis of the output test data and the informationgenerated by the supplementary data analysis element 206. For example,the supplementary data analysis element 206 may identify tests havinghigh incidences of failure or outliers, such as by comparing the totalor average number of failures, outliers, or outliers in a particularclassification to one or more threshold values.

The supplementary data analysis element 206 may also be configured tocorrelate data from different tests to identify similar or dissimilartrends, for example by comparing cumulative counts, outliers, and/orcorrelating outliers between wafers or other data sets. Thesupplementary data analysis element 206 may also analyze and correlatedata from different tests to identify and classify potential criticaland/or marginal and/or good parts on the wafer. The supplementary dataanalysis element 206 may also analyze and correlate data from differenttests to identify user-defined good part patterns and/or bad partpatterns on a series of wafers for the purposes of dynamic test timereduction.

The supplementary data analysis element 206 is also suitably configuredto analyze and correlate data from different tests to identifyuser-defined pertinent raw data for the purposes of dynamicallycompressing the test data into memory. The supplementary data analysiselement 206 may also analyze and correlate statistical anomalies andtest data results for test node in-situ setup qualification andsensitivity analysis. Further, the supplementary data analysis element206 may contribute to test node yield leveling analysis, for example byidentifying whether a particular test node may be improperly calibratedor otherwise producing inappropriate results. The supplementary dataanalysis element 206 may moreover analyze and correlate the data for thepurposes of test program optimization including, but not limited to,automatic identification of redundant tests using correlated results andoutlier analysis and providing additional data for use in analysis. Thesupplementary data analysis element 206 is also suitably configured toidentify critical tests, for example by identifying regularly failed oralmost failed tests, tests that are almost never-fail, and/or testsexhibiting a very low Cpk.

The supplementary data analysis may also provide identification of testsampling candidates, such as tests that are rarely or never failed or inwhich outliers are never detected. The supplementary data analysiselement 206 may also provide identification of the best order testsequence based on correlation techniques, such as conventionalcorrelation techniques, combined with analysis and correlation ofidentified outliers and/or other statistical anomalies, number offailures, critical tests, longest/shortest tests, or basic functionalityissues associated with failure of the test.

The supplementary data analysis may also provide identification ofcritical, marginal, and good parts as defined by sensitivity parametersin a recipe configuration file. Part identification may providedisposition/classification before packaging and/or shipping the partthat may represent a reliability risk, and/or test time reductionthrough dynamic probe mapping of bad and good parts during wafer probe.Identification of these parts may be represented and output in anyappropriate manner, for example as good and bad parts on a dynamicallygenerated prober control map (for dynamic mapping), a wafer map used foroffline inking equipment, a test strip map for strip testing at finaltest, a results file, and/or a database results table.

Supplemental data analysis at the cell controller level tends toincrease quality control at the probe, and thus final test yields. Inaddition, quality issues may be identified at product run time, notlater. Furthermore, the supplemental data analysis and signatureanalysis tends to improve the quality of data provided to the downstreamand offline analysis tools, as well as test engineers or otherpersonnel, by identifying outliers. For example, the computer 108 mayinclude information on the wafer map identifying a group of componentshaving signature analyses indicating a fault in the manufacturingprocess. Thus, the signature analysis system may identify potentiallydefective goods that may be undetected using conventional test analysis.

Referring now to FIG. 10 , an array of semiconductor devices arepositioned on a wafer. In this wafer, the general resistivity ofresistor components in the semiconductor devices varies across thewafer, for example due to uneven deposition of material or treatment ofthe wafer. The resistance of any particular component, however, may bewithin the control limits of the test. For example, the targetresistance of a particular resistor component may be 1000 Ω+/−10%. Nearthe ends of the wafer, the resistances of most of the resistorsapproach, but do not exceed, the normal distribution range of 900Ω and1100Ω (FIG. 11 ).

Components on the wafer may include defects, for example due to acontaminant or imperfection in the fabrication process. The defect mayincrease the resistance of resistors located near the low-resistivityedge of the wafer to 1080Ω. The resistance is well over the 1000Ωexpected for a device near the middle of the wafer, but is still wellwithin the normal distribution range.

Referring to FIGS. 12A-B, the raw test data for each component may beplotted. The test data exhibits considerable variation, due in part tothe varying resistivity among components on the wafer as the proberindexes across rows or columns of devices. The devices affected by thedefect are not easily identifiable based on visual examination of thetest data or comparison to the test limits.

When the test data is processed according to various aspects of thepresent invention, the devices affected by the defect may be associatedwith outliers in the test data. The test data is largely confined to acertain range of values. The data associated with the defects, however,are unlike the data for the surrounding components. Accordingly, thedata illustrate the departure from the values associated with thesurrounding devices without the defect. The outlier classificationelement 212 may identify and classify the outliers according to themagnitude of the departure of the outlier data from the surroundingdata.

The output element 208 collects data from the test system 100, suitablyat run time, and provides an output report to a printer, database,operator interface, or other desired destination. Any form, such asgraphical, numerical, textual, printed, or electronic form, may be usedto present the output report for use or subsequent analysis. The outputelement 208 may provide any selected content, including selected outputtest data from the tester 102 and results of the supplementary dataanalysis.

In the present embodiment, the output element 208 suitably provides aselection of data from the output test data specified by the operator aswell as supplemental data at product run time via the dynamic datalog.Referring to FIG. 7 , the output element 208 initially reads a samplingrange from the database 114 (step 702). The sampling range identifiespredetermined information to be included in the output report. In thepresent embodiment, the sampling range identifies components 106 on thewafer selected by the operator for review. The predetermined componentsmay be selected according to any criteria, such as data for variouscircumferential zones, radial zones, random components, or individualstepper fields. The sampling range comprises a set of x-y coordinatescorresponding to the positions of the predetermined components on thewafer or an identified portion of the available components in a batch.

The output element 208 may also be configured to include informationrelating to the outliers, or other information generated or identifiedby the supplementary data analysis element, in the dynamic datalog (step704). If so configured, the identifiers, such as x-y coordinates, foreach of the outliers are assembled as well. The coordinates for theoperator-selected components and the outliers are merged into thedynamic datalog (step 706), which in the current embodiment is in theformat of the native tester data output format. Merging resulting datainto the dynamic datalog facilitates compression of the original datainto summary statistics and critical raw data values into a smallernative tester data file, reducing data storage requirements withoutcompromising data integrity for subsequent customer analysis. The outputelement 208 retrieves selected information, such as the raw test dataand one or more data from the supplementary data analysis element 206,for each entry in the merged x-y coordinate array of the dynamic datalog(step 708).

The retrieved information is then suitably stored in an appropriateoutput report (step 710). The report may be prepared in any appropriateformat or manner. In the present embodiment, the output report suitablyincludes the dynamic datalog having a wafer map indicating the selectedcomponents on the wafer and their classification. Further, the outputelement 208 may superimpose wafer map data corresponding to outliers onthe wafer map of the preselected components. Additionally, the outputelement may include only the outliers from the wafer map or batch as thesampled output. The output report may also include a series of graphicalrepresentations of the data to highlight the occurrence of outliers andcorrelations in the data. The output report may further includerecommendations and supporting data for the recommendations. Forexample, if two tests appear to generate identical sets of failuresand/or outliers, the output report may include a suggestion that thetests are redundant and recommend that one of the tests be omitted fromthe test program. The recommendation may include a graphicalrepresentation of the data showing the identical results of the tests.

The output report may be provided in any suitable manner, for exampleoutput to a local workstation, sent to a server, activation of an alarm,or any other appropriate manner (step 712). In one embodiment, theoutput report may be provided off-line such that the output does notaffect the operation of the system or transfer to the main server. Inthis configuration, the computer 108 copies data files, performs theanalysis, and generates results, for example for demonstration orverification purposes.

In addition to the supplementary analysis of the data on each wafer, atesting system 100 according to various aspects of the present inventionmay also perform composite analysis of the data and generate additionaldata to identify patterns and trends over multiple datasets, such asusing multiple wafers and/or lots. Composite analysis is suitablyperformed to identify selected characteristics, such as patterns orirregularities, among multiple datasets. For example, multiple datasetsmay be analyzed for common characteristics that may represent patterns,trends, or irregularities over two or more datasets.

The composite analysis may comprise any appropriate analysis foranalyzing test data for common characteristics among the datasets, andmay be implemented in any suitable manner. For example, in the presenttesting system 100, the composite analysis element 214 performscomposite analysis of data derived from multiple wafers and/or lots. Thetest data for each wafer, lot, or other grouping forms a dataset. Thecomposite analysis element 214 is suitably implemented as a softwaremodule operating on the computer 108. The composite analysis element 214may be implemented, however, in any appropriate configuration, such asin a hardware solution or a combined hardware and software solution.Further, the composite analysis element 214 may be executed on the testsystem computer 108 or a remote computer, such as an independentworkstation or a third party's separate computer system. The compositeanalysis may be performed at run time, following the generation of oneor more complete datasets, or upon a collection of data generated wellbefore the composite analysis, including historical data.

The composite analysis may use any data from two or more datasets.Composite analysis can receive several sets of input data, including rawdata, filtered data, and/or smoothed data, for each dataset, such as byexecuting multiple configurations through the classification engine.Once received, the input data is suitably filtered using a series ofuser-defined rules, which can be defined as mathematical expressions,formulas, or any other criteria.

In one embodiment, the input data may be filtered using a data-sortingneural network. In one embodiment, the data-sorting neural network maybe selected from a library of data-sorting neural networks. The libraryof data-sorting neural networks may comprise a database of previouslytrained data-sorting neural networks. The previously traineddata-sorting neural networks may comprise data-sorting neural networksthat are optimized for various situations. In one embodiment, thecomposite analysis may be configured to automatically select adata-sorting neural network from the library of data-sorting neuralnetworks based on the test data or the type of test.

In one embodiment, the data-sorting neural network may comprise a newlytrained data-sorting neural network. A user may select to train a newdata-sorting neural network or, in the absence of the library ofdata-sorting neural networks, the system may automatically train the newdata-sorting neural network.

In one embodiment, the neural network may be trained using data andtargets to represent the likelihood that the tests/wafers exhibit theappropriate characteristics of the various sorting classes. For example,if raw data is characterized by an integer representing a test scoreranging between 0 and 31, then the neural network may identify that aninput data with a value of 24 corresponds to a raw data value and thedata should be filtered accordingly.

In one embodiment, the data-sorting neural network may comprise aself-learning neural network such as a data-sorting self-organizing map(SOM). In one embodiment, a new data-sorting SOM's training starts bythe user identifying a number of SOM inputs. A SOM input may comprise afeature that may be used to describe the input data. The user must alsoprovide a number of SOM outputs. The SOM outputs may compriseclassification classes that the input data will be sorted into.

The SOM may further comprise neurons. Each neuron may receive input fromthe SOM inputs and be associated with one or more SOM outputs. Theneuron may comprise a node and a weighted vector. The weighted vectormay comprise a n^(th) dimension vector where n is equal to the number ofSOM inputs. The SOM neurons may be classified as a winning neuron, asneighboring neurons, and as non-neighboring neurons. The winning neuronmay comprise a neuron that most closely resembles the SOM input. In oneembodiment, the neuron that most closely resembles the SOM input may befound by comparing a Euclidean distances of all the weighted vectors tothe SOM input. The neuron with closest Euclidean distance is deemed tobe the winning neuron.

In one embodiment, the SOM may be trained using sample input data. Afterthe number of SOM inputs and SOM outputs have been selected, the SOMinputs may be provided with input data. Initially, the weighted vectorof each neuron will be set to a default weight. The default weight maycomprise a random weight. For each input data a winning neuron isselected. The neuron's weighted vector may then be updated. Updating theneuron's weighted vector may comprise adjusting the weighted vector tomore closely resemble the input data.

The neuron that has a weighted vector that is close in value to thewinning neuron's weighted vector may be referred to as a neighboringneuron. The neighboring neurons may also be updated when a winningneuron is updated. In one embodiment, the neighboring neurons may beupdated at a lower rate than the winning neuron. In one embodiment, theneighboring neurons may be updated at a rate that is dependent on howclose the neuron is to the winning neuron. For example, a neighboringneuron that is relatively close to the winning neuron may be updated athalf the rate of the winning neuron while another neighboring neuronthat is not as close to the winning neuron may only be updated at onequarter of the rate of the winning neuron.

During training, each input data may result in the updating of thewinning neuron and the neighboring neurons. In one embodiment, the rateat which the winning neuron and the neighboring neurons are updated maybe progressively reduced as training progressive. Multiple iterationsmay be used to create groups of similar neurons called classneighborhoods. Thus, over time, the neurons may become selectively tunedto various patterns or classes of patterns present in the input data dueto the competitive learning process.

After the training has been completed, the SOM may be saved to the SOMlibrary, it may be used on the input data, or it may be discarded and adifferent SOM may be selected or trained. In operation, each wafer willbe classified into one of the classes and no additional classes will beformed.

In one embodiment, the data-sorting SOM may be trained with the inputdata that is representative of the various data classes including rawdata, filtered data, and/or smoothed data. For example, if there arethree different types of input data to be sorted by the SOM, the SOM maybe trained to classify the input into the three classes by providing theSOM with representative data from each type of input data. Aftertraining has been completed, the SOM may operate as a classifier. Thus,the resulting SOM may classify all the input the SOM receives into oneof these classes.

After the input data has been suitably filtered, the data may beanalyzed to identify patterns or irregularities in the data. Thecomposite data may also be merged into other data, such as the raw dataor analyzed data, to generate an enhanced overall dataset. The compositeanalysis element 214 may then provide an appropriate output report thatmay be used to improve the test process. For example, the output reportmay provide information relating to issues in a manufacturing and/ortesting process.

In the present system, the composite analysis element 214 analyzes setsof wafer data in conjunction with user expressions or other suitableprocesses and a spatial analysis to build and establish composite mapsthat illustrate significant patterns or trends. Composite analysis canreceive several different datasets and/or composite maps for any one setof wafers by executing multiple user configurations on the set ofwafers.

Referring to FIG. 13 , in the present embodiment operating in thesemiconductor testing environment, the composite analysis element 214receives data from multiple datasets, such as the data from multiplewafers or lots (1310). The data may comprise any suitable data foranalysis, such as raw data, filtered data, smoothed data, historicaldata from prior test runs, or data received from the tester at run time.In the present embodiment, the composite analysis element 214 receivesraw data and filtered data at run time. The filtered data may compriseany suitable data for analysis, such as smoothed data and/or signatureanalysis data. In the present embodiment, the composite analysis element214 receives the raw dataset and supplementary data generated by thesupplementary data analysis element 206, such as the filtered data,smoothed data, identification of failures, identification of outliers,signature analysis data, and/or other data.

After receiving the raw data and the supplementary data, the compositeanalysis element 214 generates composite data for analysis (1312). Thecomposite data comprises data representative of information from morethan one dataset. For example, the composite data may comprise summaryinformation relating to the number of failures and/or outliers for aparticular test occurring for corresponding test data in differentdatasets, such as data for components at identical or similar positionson different wafers or in different lots. The composite data may,however, comprise any appropriate data, such as data relating to areashaving concentrations of outliers or failures, wafer locationsgenerating a significant number of outliers or failures, or other dataderived from two or more datasets.

The composite data is suitably generated by comparing data from thevarious datasets to identify patterns and irregularities among thedatasets. For example, the composite data may be generated by ananalysis engine configured to provide and analyze the composite dataaccording to any suitable algorithm or process. In the presentembodiment, the composite analysis element 214 includes a proximityengine configured to generate one or more composite masks based on thedatasets. The composite analysis element 214 may also process thecomposite mask data, for example to refine or emphasize information inthe composite mask data.

In the present embodiment, the proximity engine receives multipledatasets, either through a file, memory structure, database, or otherdata store, performs a spatial analysis on those datasets (1320), andoutputs the results in the form of a composite mask. The proximityengine may generate the composite mask data, such as a composite imagefor an overall dataset, according to any appropriate process ortechnique using any appropriate methods. In particular, the proximityengine suitably merges the composite data with original data (1322) andgenerates an output report for use by the user or another system (1324).The proximity engine may also be configured to refine or enhance thecomposite mask data for analysis, such as by spatial analysis, analyzingthe data for recurring characteristics in the datasets, or removing datathat does not meet selected criteria.

In the present embodiment, the proximity engine performs composite maskgeneration 1312, and may also be configured to determine exclusion areas1314, perform proximity weighting 1316, and detect and filter clusters1318. The proximity engine may also provide proximity adjustment orother operations using user-defined rules, criteria, thresholds, andprecedence. The result of the analysis is a composite mask of theinputted datasets that illustrates spatial trends and/or patterns foundthroughout the datasets given. The proximity engine can utilize anyappropriate output method or medium, including memory structures,databases, other applications, and file-based data stores such as textfiles or XML files in which to output the composite mask.

The proximity engine may use any appropriate technique to generate thecomposite mask data, including cumulative squared methods, N-pointsformulas, Western Electrical rules, or other user defined criteria orrules. In the present embodiment, composite mask data may be consideredas an overall encompassing or “stacked” view of multiple datasets. Thepresent proximity engine collects and analyzes data for correspondingdata from multiple datasets to identify potential relationships orcommon characteristics in the data for the particular set ofcorresponding data. The data analyzed may be any appropriate data,including the raw data, filtered data, smoothed data, signature analysisdata, and/or any other suitable data.

In the present embodiment, the proximity engine analyzes data forcorresponding locations on multiple wafers. Referring to FIG. 14 , eachwafer has devices in corresponding locations that may be designatedusing an appropriate identification system, such as an x, y coordinatesystem. Thus, the proximity engine compares data for devices atcorresponding locations or data points, such as location 10, 12 as shownin FIG. 14 , to identify patterns in the composite set of data.

The proximity engine of the present embodiment uses at least one of twodifferent techniques for generating the composite mask data, acumulative squared method and a formula-based method. The proximityengine suitably identifies data of interest by comparing the data toselected or calculated thresholds. In one embodiment, the proximityengine compares the data points at corresponding locations on thevarious wafers and/or lots to thresholds to determine whether the dataindicate potential patterns across the datasets. The proximity enginecompares each datum to one or more thresholds, each of which may beselected in any appropriate manner, such as a predefined value, a valuecalculated based on the current data, or a value calculated fromhistorical data.

For example, a first embodiment of the present proximity engineimplements a cumulative squared method to compare the data tothresholds. In particular, referring to FIG. 15 , the proximity enginesuitably selects a first data point (1512) in a first dataset (1510),such as a result for a particular test for a particular device on aparticular wafer in a particular lot, and compares the data point valueto a count threshold (1514). The threshold may comprise any suitablevalue, and any type of threshold, such as a range, a lower limit, anupper limit, and the like, and may be selected according to anyappropriate criteria. If the data point value exceeds the threshold,i.e., is lower than the threshold, higher than the threshold, within thethreshold limits, or whatever the particular qualifying relationship maybe, an absolute counter is incremented (1516) to generate a summaryvalue corresponding to the data point.

The data point value is also compared to a cumulative value threshold(1518). If the data point value exceeds the cumulative value threshold,the data point value is added to a cumulative value for the data point(1520) to generate another summary value for the data point. Theproximity engine repeats the process for every corresponding data point(1522) in every relevant dataset (1524), such as every wafer in the lotor other selected group of wafers. Any other desired tests orcomparisons may be performed as well for the data points and datasets.

When all of the relevant data points in the population have beenprocessed, the proximity engine may calculate values based on theselected data, such as data exceeding particular thresholds. Forexample, the proximity engine may calculate overall cumulativethresholds for each set of corresponding data based on the cumulativevalue for the relevant data point (1526). The overall cumulativethreshold may be calculated in any appropriate manner to identifydesired or relevant characteristics, such as to identify sets ofcorresponding data points that bear a relationship to a threshold. Forexample, the overall cumulative threshold (Limit) may be definedaccording to the following equation:

${Limit} = {{Average} + \frac{\left( {{ScaleFactor}*{Standard}{Deviation}^{2}} \right)}{\left( {{Max} - {Min}} \right)}}$where Average is the mean value of the data in the composite populationof data, Scale Factor is a value or variable selected to adjust thesensitivity of the cumulative squared method, Standard Deviation is thestandard deviation of data point values in the composite population ofdata, and (Max−Min) is the difference between the highest and lowestdata point values in the complete population of data. Generally, theoverall cumulative threshold is defined to establish a comparison valuefor identifying data points of interest in the particular data set.

Upon calculation of the overall cumulative threshold, the proximityengine determines whether to designate each data point for inclusion inthe composite data, for example by comparing the count and cumulativevalues to thresholds. The proximity engine of the present embodimentsuitably selects a first data point (1528), squares the total cumulativevalue for the data point (1530), and compares the squared cumulativevalue for the data point to the dynamically generated overall cumulativethreshold (1532). If the squared cumulative value exceeds the overallcumulative threshold, then the data point is designated for inclusion inthe composite data (1534).

The absolute counter value for the data point may also be compared to anoverall count threshold (1536), such as a pre-selected threshold or acalculated threshold based on, for example, a percentage of the numberof wafers or other datasets in the population. If the absolute countervalue exceeds the overall count threshold, then the data point may againbe designated for inclusion in the composite data (1538). The process issuitably performed for each data point (1540).

The proximity engine may also generate the composite mask data usingother additional or alternative techniques. The present proximity enginemay also utilize a formula-based system for generating the compositemask data. A formula-based system according to various aspects of thepresent invention uses variables and formulas, or expressions, to definea composite wafer mask.

For example, in an exemplary formula-based system, one or more variablesmay be user-defined according to any suitable criteria. The variablesare suitably defined for each data point in the relevant group. Forexample, the proximity engine may analyze each value in the datapopulation for the particular data point, for example to calculate avalue for the data point or count the number of times a calculationprovides a particular result. The variables may be calculated for eachdata point in the dataset for each defined variable.

After calculating the variables, the data points may be analyzed, suchas to determine whether the data point meets the user-defined criteria.For example, a user-defined formula may be resolved using the calculatedvariable values, and if the formula equates to a particular value orrange of values, the data point may be designated for inclusion in thecomposite mask data.

Thus, the proximity engine may generate a set of composite mask dataaccording to any suitable process or technique. The resulting compositemask data comprises a set of data that corresponds to the results of thedata population for each data point. Consequently, characteristics forthe data point may be identified over multiple datasets. For example, inthe present embodiment, the composite mask data may illustrateparticular device locations that share characteristics on multiplewafers, such as widely variable test results or high failure rates. Suchinformation may indicate issues or characteristics in the manufacturingor design process, and thus may be used to improve and controlmanufacturing and testing.

The composite mask data may also be analyzed to generate additionalinformation. For example, the composite mask data may be analyzed toillustrate spatial trends and/or patterns in the datasets and/oridentify or filter significant patterns, such as filtering to reduceclutter from relatively isolated data points, enhancing or refiningareas having particular characteristics, or filtering data having knowncharacteristics. The composite mask data of the present embodiment, forexample, may be subjected to spatial analyses to smooth the compositemask data and complete patterns in the composite mask data. Selectedexclusion zones may receive particular treatment, such that compositemask data may be removed, ignored, enhanced, accorded lowersignificance, or otherwise distinguished from other composite mask data.A cluster detection process may also remove or downgrade the importanceof data point clusters that are relatively insignificant or unreliable.

In the present embodiment, the proximity engine may be configured toidentify particular designated zones in the composite mask data suchthat data points from the designated zones are accorded particulardesignated treatment or ignored in various analyses. For example,referring to FIG. 16 , the proximity engine may establish an exclusionzone at a selected location on the wafers, such as individual devices,groups of devices, or a band of devices around the perimeter of thewafer. The exclusion zone may provide a mechanism to exclude certaindata points from affecting other data points in proximity analysisand/or weighting. The data points are designated as excluded in anysuitable manner, such as by assigning values that are out of the rangeof subsequent processes.

The relevant zone may be identified in any suitable manner. For example,excluded data points may be designated using a file listing of deviceidentifications or coordinates, such as x,y coordinates, selecting aparticular width of band around the perimeter, or other suitable processfor defining a relevant zone in the composite data. In the presentembodiment, the proximity engine may define a band of excluded deviceson a wafer using a simple calculation that causes the proximity engineto ignore or otherwise specially treat data points within a user definedrange of the edge of the data set. For example, all devices within thisrange, or listed in the file, are then subject to selected exclusioncriteria. If the exclusion criteria are met, the data points in theexclusion zone or the devices meeting the exclusion criteria areexcluded from one or more analyses.

The proximity engine of the present embodiment is suitably configured toperform additional analyses upon the composite mask data. The additionalanalyses may be configured for any appropriate purpose, such as toenhance desired data, remove unwanted data, or identify selectedcharacteristics in the composite mask data. For example, the proximityengine is suitably configured to perform a proximity weighting process,such as based on a point weighting system, to smooth and completepatterns in the composite mask data.

Referring to FIGS. 17A-B and 18, the present proximity engine searchesthrough all data points in the dataset. The proximity engine selects afirst point (1710) and checks the value of the data point against acriterion, such as a threshold or a range of values (1712). When a datapoint is found that exceeds the selected threshold or is within theselected range, the proximity engine searches data points surroundingthe main data point for values (1714). The number of data points aroundthe main data point may be any selected number, and may be selectedaccording to any suitable criteria.

The proximity engine searches the surrounding data points for datapoints exceeding an influence value or satisfying another suitablecriterion indicating that the data point should be accorded weight(1716). If the data point exceeds the influence value, the proximityengine suitably assigns a weight to the main data point according to thevalues of the surrounding data points. In addition, the proximity enginemay adjust the weight according to the relative position of thesurrounding datapoint. For example, the amount of weight accorded to asurrounding data point can be determined according to whether thesurrounding data point is adjacent (1718) or diagonal (1720) to the maindata point. The total weight may also be adjusted if the data point ison the edge of the wafer (1722). When all surrounding data points aroundthe main data point have been checked (1724), the main data point isassigned an overall weight, for example by adding the weight factorsfrom the surrounding data points. The weight for the main data point maythen be compared to a threshold, such as a user defined threshold(1726). If the weight meets or exceeds the threshold, the data point isso designated (1728).

The composite mask data may also be further analyzed to filter data. Forexample, in the present embodiment, the proximity engine may beconfigured to identify, and suitably remove, groups of data points thatare smaller than a threshold, such as a user-specified threshold.Referring to FIGS. 19 and 20 , the proximity engine of the presentembodiment may be configured to define the groups, size them, and removesmaller groups. To define the groups, the proximity engine searchesthrough every data point in the composite mask data for a data pointsatisfying a criterion. For example, the data points in the compositemask data may be separated into ranges of values and assigned indexnumbers. The proximity engine begins by searching the composite maskdata for a data point matching a certain index (1910). Upon encounteringa data point meeting designated index (1912), the proximity enginedesignates the found point as the main data point and initiates arecursive program that searches in all directions from the main datapoint for other data points that are in the same index, oralternatively, have substantially the same value, also exceed thethreshold, or meet other desired criteria (1914).

As an example of a recursive function in the present embodiment, theproximity engine may begin searching for data points having a certainvalue, for instance five. If a data point with a value of five is found,the recursive program searches all data points around the main deviceuntil it finds another data point with the value of five. If anotherqualifying data point is found, the recursive program selects theencountered data point as the main data point and repeats the process.Thus, the recursive process analyzes and marks all data points havingmatching values that are adjacent or diagonal to each other and thusform a group. When the recursive program has found all devices in agroup having a certain value, the group is assigned a unique group indexand the proximity engine again searches through the entire compositemask data. When all of the data values have been searched, the compositemask data is fully separated into groups of contiguous data pointshaving the same group index.

The proximity engine may determine the size of each group. For example,the proximity engine may count the number of data points in the group(1916). The proximity engine may then compare the number of data pointsin each group to a threshold and remove groups that do not meet thethreshold (1918). The groups may be removed from the grouping analysisin any suitable manner (1920), such as by resetting the index value forthe relevant group to a default value. For example, if the thresholdnumber of data points is five, the proximity engine changes the groupindex number for every group having fewer than five data points to adefault value. Consequently, the only groups that remain classified withdifferent group indices are those having five or more data points.

The proximity engine may perform any appropriate additional operationsto generate and refine the composite mask data. For example, thecomposite mask data, including the results of the additional filtering,processing, and analyzing of the original composite mask data, may beused to provide information relating to the multiple datasets and theoriginal source of the data, such as the devices on the wafers or thefabrication process. The data may be provided to the user or otherwiseused in any suitable manner. For example, the data may be provided toanother system for further analysis or combination with other data, suchas executing user-defined rules combined with a merging operation on thecomposite mask data, the raw data, and any other appropriate data toproduce a data set that represents trends and patterns in the raw data.Further, the data may be provided to a user via an appropriate outputsystem, such as a printer or visual interface.

In the present embodiment, for example, the composite mask data iscombined with other data and provided to the user for review. Thecomposite mask data may be combined with any other appropriate data inany suitable manner. For example, the composite mask data may be mergedwith signature data, the raw data, hardware bin data, software bin data,and/or other composite data. The merging of the datasets may beperformed in any suitable manner, such as using various user-definedrules including expressions, thresholds, and precedence.

In the present system, the composite analysis element 214 performs themerging process using an appropriate process to merge the composite maskdata with an original map of composite data, such as a map of compositeraw data, composite signature data, or composite bin data. For example,referring to FIG. 21 , the composite analysis element 214 may merge thecomposite mask data with the original individual wafer data using anabsolute merge system in which the composite mask data is fully mergedwith the original data map. Consequently, the composite mask data ismerged with the original data map regardless of overlap or encompassmentof existing patterns. If only one composite mask illustrates a patternout of multiple composite masks, the pattern is included in the overallcomposite mask.

Alternatively, the composite analysis element 214 may merge the data inconjunction with additional analysis. The composite analysis element 214may filter data that may be irrelevant or insignificant. For example,referring to FIG. 22 , the composite analysis element 214 may merge onlydata in the composite mask data that overlaps data in the original datamap or in another composite mask, which tends to emphasize potentiallyrelated information.

The composite analysis element 214 may alternatively evaluate thecomposite mask data and the original data to determine whether aparticular threshold number, percentage, or other value of data pointsoverlap between maps. Depending on the configuration, the data mergedmay only include areas where data points, in this case corresponding todevices, overlapped sufficiently between the composite mask data and theoriginal data to meet the required threshold value for overlapping datapoints. Referring to FIG. 23 , the composite analysis element 214 may beconfigured to include only composite data patterns that overlaps to asufficient degree with tester bin failures, i.e., failed devices, in theoriginal data, such as 50% of the composite data overlapping with testerbin failure data. Thus, if less than the minimum amount of compositedata overlaps with the original data, the composite data pattern may beignored. Similarly, referring to FIG. 24 , the composite analysiselement 214 may compare two different sets of composite data, such asdata from two different recipes, and determine whether the overlapbetween the two recipes satisfies selected criteria. Only the data thatoverlaps and/or satisfies the minimum criteria is merged.

The merged data may be provided to the output element 208 for output tothe user or other system. The merged data may be passed as input toanother process, such as a production error identification process or alarge trend identification process. The merged data may also beoutputted in any assortment or format, such as memory structures,database tables, flat text files, or XML files.

In the present embodiment, the merged data and/or wafer maps areprovided into an ink map generation engine. The ink map engine producesmaps for offline inking equipment. In addition to offline inking maps,the merged data results may be utilized in generating binning resultsfor inkless assembly of parts, or any other process or application thatutilizes these types of results.

The test system 100 may also be configured to use test data to identifycharacteristics and/or problems associated with the fabrication process,including the manufacturing and/or testing process. For example, thetest system 100 may analyze test data from one or more sources andautomatically associate characteristics of the test data with knownproblems, issues, or characteristics in the manufacturing and testprocess, such as residue on pads, probing faults, conductor bridging,contamination, scratches, parametric variations, and/or stepper orreticle problems. If the test data characteristics do not correspond toa known issue, then the test system 100 may receive and storeinformation relating to the issue after it is diagnosed, such that thetest system 100 is updated to diagnose new test data characteristics asthey are encountered.

In particular, the diagnostic system 216 is suitably configured toautomatically identify test data characteristics and/or classify suchcharacteristics according to their probable source or cause. The testsystem 100 may also automatically provide an alert when a faultdetection occurs, such as an immediate failure classification andnotification at run-time and/or a later output report. Informationrelating to the sources or causes of various test data characteristicsmay be stored and compared to the data. The classification criteria andprocedures are configurable, such that when different test datacharacteristics are associated with different problems, the informationis suitably provided to the diagnostic system 216 for use in subsequentanalyses. The stored information facilitates a configurable knowledgebase which may be changed or updated according to the particular dataenvironment. The stored information also facilitates recognition ofknown scenarios and classifications for objective analysis andclassification to report probable issues based on the test framework,rules, and symptoms. As the diagnostic system 216 is updated as newpatterns are associated with particular problems, the diagnostic system216 facilitates the capture and retention of product engineeringexpertise, generates a historical database, and provides for consistent,repeatable analysis methodologies.

For example, the test data diagnostic system 216 may be configured todiagnose problems guided, at least in part, by the test data. The datamay be received and analyzed at run time, retrieved from a storagesystem after completion of one or more test runs, and/or includehistoric data. The diagnostic system 216 may receive test data from anysuitable source, such as parametric test, metrologic, process control,microscopy, spectroscopy, defect analysis, and fault isolation data. Thediagnostic system 216 may also receive processed data, such as smootheddata, filtered composite data, and additional data that is generatedbased on the test data, such as bin results, SPC data, spatial analysisdata, outlier data, composite data, and data signatures.

For example, referring to FIG. 25 , the diagnostic system 216 of thepresent embodiment is configured to analyze multiple types of data. Thediagnostic system 216 analyzes raw electronic wafer sort (EWS) data2512, as well as EWS bin signature data 2514, bin map and/or yieldpattern data 2518, outlier signature data 2520, and process control orelectrical test (ET) data 2516 for each wafer. The EWS bin signaturedata 2514 may comprise any suitable classification data based on the EWSresults, for example as may be generated by the supplementary dataanalysis element 206. In the present embodiment, the EWS bin signaturedata 2514 comprises data corresponding to each device on a waferindicating the magnitude of the device's failure (if the device did notpass), such as classifications of gross, significant, or marginal, asdetermined by the supplementary data analysis element 206.

The diagnostic system 216 also receives process control data 2516, suchas data relating to the electrical characteristics for various points onthe wafer and/or for the components 106. Further, the diagnostic system216 may receive the bin map data 2518 for the wafer indicating thepass/fail binning classifications for the components 106. In addition,the diagnostic system 216 may receive the outlier signature bin map 2520for the wafer, for example data generated by the outlier classificationelement 212. For example, each outlier in the data may be classified astiny, small, medium, or critical according to selected criteria.

The diagnosis system 216 may be configured in any suitable manner toanalyze the received data to identify process characteristics, such asproblems or issues in the manufacturing or test process. The processcharacteristics may be identified according to any suitable criteria orprocess. For example, referring to FIG. 26 , the diagnostic system 216of the present embodiment comprises a rules-based analyzer 2610 foridentifying process characteristics according to predefined criteria.Additionally or alternatively, the diagnostic system 216 may comprise apattern recognition system 2612 for identifying process characteristicsbased on patterns recognized in the test data.

In particular, the rules-based analyzer 2610 may analyze the test datafor particular characteristics corresponding to particular problemsbased on a set of definite rules. The particular characteristicssuitably comprise any known set of data corresponding to particular testor manufacturing issues. The rules-based analyzer 2610 is suitablyconfigured to analyze the data for selected types of data and generate acorresponding signal.

The pattern recognition system 2612 is suitably configured to receivethe data from the various sources and identify patterns in the data. Thepattern recognition system 2612 is also suitably configured to match theidentified patterns with known issues associated with such patterns, forexample by assigning a likelihood of a particular cause based on theidentified patterns. For example, clusters of devices having similarnon-passing bin results or outliers located in the same position ondifferent wafers may indicate a particular problem in the manufacturingprocess. The pattern recognition system 2612 identifies and analyzespatterns in the data that may indicate such issues in the manufacturingand/or test process.

The pattern recognition system 2612 may be configured in any suitablemanner to identify patterns in the various test data and analyze thepatterns for correspondence to potential manufacturing or test issues.In the present embodiment, the pattern recognition system 2612 comprisesan intelligent system configured to recognize patterns, such as spatialpatterns of clustered defects or outliers, in the test data. Inparticular, the pattern recognition system 2612 of the presentembodiment includes a pattern identifier 2614 and a classifier 2616. Thepattern identifier 2614 processes the data for handling by theclassifier 2616 and/or identifies patterns in the received data that maycorrespond to issues. The classifier 2616 classifies the identifiedpatterns or other data to different known categories or an unknowncategory.

In one embodiment, the classifier 2616 may comprise a classifying neuralnetwork configured to analyze and classify the test data. In oneembodiment, the classifying neural network may be selected from alibrary of classifying neural networks. The library of classifyingneural networks may comprise a database of previously trainedclassifying neural networks. The previously trained classifying neuralnetworks may comprise classifying neural networks that are optimized forvarious situations. In one embodiment, the classifier may be configuredto automatically select a classifying neural network from the library ofclassifying neural networks based on the test data or the type of test.

In one embodiment, the classifying neural network may comprise a newlytrained classifying neural network. A user may select to train a newclassifying neural network or, in the absence of the library ofclassifying neural networks, the system may automatically train the newclassifying neural network.

In one embodiment, the classifying neural network may be trained usingdata and targets to represent the likelihood that the tests/wafersexhibit the appropriate characteristics of the various sorting classes.In one embodiment, the classifying neural network may be trained usingbin-level data and/or parametric test data.

In one embodiment, training data is supplied to the neural network.Training data may comprise target values, feature values, binmaps, andindexing values and may include entire datasets or a sample of thedatasets. In one embodiment of the invention, the training data maycomprise a random selection of a portion of an available training data.For example, in one embodiment the training data may comprise a portionof the available data. In one embodiment, the training data is absent ofa pattern.

The training data may comprise nominal dataset features. In oneembodiment, the nominal dataset features may be normalized so that thenominal features have a mean value equal to zero and a standarddeviation equal to one. A mean and a standard deviation may then becalculated for all of the features. Upper and lower dataset limits maybe calculated and the entire dataset may be compared for each of thefeatures being analyzed with the limits. For example, in one embodimentof the invention, the upper and lower dataset limits may be set at themean±two standard deviations and may also be user adjustable. If data isoutside of one of these limits for a feature, the wafers/tests may beset as a patterned wafers/tests.

In one embodiment, the classifying neural network may comprise aself-learning neural network such as a classifying self-organizing map(SOM). In another embodiment, a classifying SOM or other self-learningneural network may be used in conjunction with the classifying neuralnetwork to classify the test data.

In one embodiment, the classifying SOM may be selected from a library ofclassifying SOMs. The library of classifying SOMs may comprise adatabase of previously trained classifying SOMs. The previously trainedclassifying SOMs may comprise classifying SOMs that are optimized forvarious situations. In one embodiment, the classifier 2616 may beconfigured to automatically select a classifying SOM from the library ofclassifying SOMs based on the test data or the type of test.

In one embodiment, a new classifying SOM's training starts by the useridentifying a number of SOM inputs. A SOM input may comprise a featurethat may be used to describe the input data. In one embodiment, thefeatures may be defined by the feature extractor 2620. The user mustalso provide a number of SOM outputs. The SOM outputs may compriseclassification classes that the input data will be sorted into.

The SOM may further comprise neurons. Each neuron may receive input fromthe SOM inputs and be associated with one or more SOM outputs. Theneuron may comprise a node and a weighted vector. The weighted vectormay comprise a n^(th) dimension vector where n is equal to the number ofSOM inputs. The SOM neurons may be classified as a winning neuron, asneighboring neurons, and as non-neighboring neurons. The winning neuronmay comprise a neuron that most closely resembles the SOM input. In oneembodiment, a neuron that most closely resembles the SOM input may befound by comparing a Euclidean distances of all the weighted vectors tothe SOM input. The neuron with closest Euclidean distance is deemed tobe the winning neuron.

In one embodiment, the SOM may be trained using sample input data. Afterthe number of SOM inputs and SOM outputs have been selected, the SOMinputs may be provided with input data. Initially, the weighted vectorof each neuron will be set to a default weight. The default weight maycomprise a random weight. For each input data a winning neuron isselected. The neuron's weighted vector may then be updated. Updating theneuron's weighted vector may comprise adjusting the weighted vector tomore closely resemble the input data.

A neuron that has a weighted vector that is close in value to thewinning neuron's weighted vector may be referred to as a neighboringneuron. The neighboring neurons may also be updated when a winningneuron is updated. In one embodiment, the neighboring neurons may beupdated at a lower rate than the winning neuron. In one embodiment, theneighboring neurons may be updated at a rate that is dependent on howclose the neuron is to the winning neuron. For example, a neighboringneuron that is relatively close to the winning neuron may be updated athalf the rate of the winning neuron while another neighboring neuronthat is not as close to the winning neuron may only be updated at onequarter of the rate of the winning neuron.

During training, each input data may result in the updating of thewinning neuron and the neighboring neurons. In one embodiment, the rateat which the winning neuron and the neighboring neurons are updated maybe progressively reduced as training progressive. Multiple iterationsmay be used to create groups of similar neurons called classneighborhoods. Thus, over time, the neurons may become selectively tunedto various patterns or classes of patterns present in the input data dueto the competitive learning process.

After the training has been completed, the SOM may be saved to the SOMlibrary, it may be used on the input data, or it may be discarded and adifferent SOM may be selected or trained. In operation, each wafer willbe classified into one of the classes and no additional classes will beformed.

In one embodiment, the SOM classes may be pre-defined classes that maybe defined during the training of the neural network. For example,during the training of the neural network a number of classes may bedesignated at the initiation of training. As training progresses, theneural network may group similar input data. From these data groupings,distinct classes may be formed. If specific classes are desired, a usermay supply the neural network representative data from those classesduring training. For example, if one of the output classes is defined bya specific pattern of defects such as a crescent pattern, then datashowing a crescent pattern may be provided during training.

The pattern identifier 2614 may perform different operations ondifferent types of data. For example, the pattern identifier 2614 maynot process certain data at all, such as EWS bin signature data or otherdata that does not need preliminary processing to be used by theclassifier 2616. Other test results may be subjected to statisticalanalysis. For example, the pattern identifier 2614 may perform aprincipal components analysis on the EWS test results, which generatesone or more eigenvectors and or eigenvalues. The principal componentsanalysis may be based on covariance of any appropriate variables, suchas wafer location, test type, and wafer sequence. In the presentembodiment, the pattern identifier 2614 may select an eigenvectorassociated with higher eigenvalues, or just the eigenvector associatedwith the highest eigenvalues, and ignores the components with lessersignificance.

Other data may be summarized or reduced. For example, the processcontrol data may be summarized as a series of test value sumscorresponding to the test structures on the wafer. Thus, the patternidentifier 2614 sums the values for each test for each test structure.The number of sums (and thus the size of the vector) is thereforeidentical to the number of test structures.

Other data may also be filtered and further processed by the patternidentifier 2614. For example, the pattern identifier 2614 may beconfigured in any suitable manner to filter noise or other undesireddata from the data set and/or identify patterns in the test data, suchas the bin map data and the outlier signature bin map. For example, thepattern identifier 2614 is suitably configured to filter out data setsthat exhibit no patterns, generate data relating to the patterns in thedata, and select particular patterns or other characteristics forclassification. In the present embodiment, the pattern identifier 2614comprises a pattern filter 2618, a feature extractor 2620, and a featureselector 2622. The pattern filter 2618 determines whether a data set,such as the data for a particular wafer, includes a pattern. The featureextractor 2620 generates features that exhibit information for the datasets designated by the pattern filter 2618 and are suitable for analysisby the classifier 2616. The feature selector 2622 analyzes the generatedfeatures according to selected criteria to select features forclassification.

The pattern filter 2618 may be configured to identify patterns in thedata set in any suitable manner. For example, the pattern filter 2618suitably comprises a software module configured to process the receiveddata to detect whether any patterns are in the data. To preserve theintegrity of the test data, the pattern filter suitably processes thedata without losing information from the original data. The patternfilter 2618 may discard data sets without patterns, leaving only datasets having detected patterns. The pattern filter 2618 may be configuredto analyze the various types of data individually or in combination. Thepattern filter 2618 may identify any appropriate patterns, such asbulls-eye, hot spot, ring, and other patterns.

In the present embodiment, the pattern filter 2618 analyzes the dataaccording to a pattern mining algorithm and in conjunction with one ormore masks associated with known patterns or theoretical patterns. Forexample, referring to FIG. 28 , the pattern filter may be configured toperform median filtering of the test data using a two-dimensionale-binmap in conjunction with the pattern mask. The pattern mask maycomprise any suitable mask, which determines which devices in thee-binmap are selected to perform the median filtering. For example, thepatterns may be defined from pre-existing, real-world scenarios, or bysimulations generated from domain experts. The pattern mask is suitablysimilar to the pattern to be identified by the classifier 2616. Forexample, the pattern mask may utilize the information generated by thecomposite analysis element 214, such as the composite mask data fromvarious data sets or merged composite mask data, though any appropriatesimulated theoretical pattern may be used.

The median filtering is performed around each value that has beenselected by the mask in the original e-binmap data. In particular, eachdata point in the data set and selected data points surrounding eachdata point are compared to each selected mask. For example, around eachvalue that is selected by the mask in the test data, the median iscalculated considering a neighborhood of a size n-by-n, such as athree-by-three window. Those data sets that fail to exhibit a patternare ignored. Those that include a pattern are provided to the featureextractor 2620. Alternative filtering techniques may also be applied,such as the proximity analysis pattern isolation techniques disclosedherein.

In the present embodiment, the pattern filter 2618 also reduces noise inthe data set, for example to remove intermittent noise from the testdata. The pattern filter 2618 may employ any suitable system forfiltering noise, such as spatial filtering, median filtering, or aconvolution process on the test data. In one exemplary embodiment, thepattern filter 2618 utilizes spatial filtering, such as cross-shapemedian filtering, to reduce noise, such as “salt-and-pepper” noise dueto outliers in the data.

Data sets having patterns may be analyzed to match the identifiedpatterns to particular issues. Under certain circumstances, however, rawdata used by the pattern filter 2618 may not be suitable for analysis bythe classifier 2616. Consequently, the feature extractor 2620 may createdata, based on the test data, that may be used by the classifier 2616.

In the present embodiment, the feature extractor 2620 generates featuresthat exhibit information from the data sets designated by the patternfilter 2618. The features may be particularly useful in situations inwhich the original data is difficult or impossible to use. The featuresmay then be analyzed by the classifier 2616 to identify the type ofpattern identified by the pattern filter 2618. For example, the featureextractor 2620 may be configured to encode relevant information in theoriginal data, such as by computing values for a set of variables basedon the data set. The features are suitably configured to efficientlyencode the relevant information residing in the original data and beused by the classifier 2616 to classify the corresponding patterns inthe data set into fault classes. The features are also suitablycalculated from any set of data, and are thus independent of theparticular components under test to the nature of the tests themselves.

The features may comprise any appropriate information extracted from thedata. In the present embodiment, the feature extractor 2620 calculatesseveral features that present normalized and/or condensed datasubstantially regardless of the particular device being tested orcharacteristics of the data set, such as a mass, a centroid, a geometricset of moments, and seven moments of Hu from the test data. The massprovides information regarding the size of the distribution within adata set of interest. The centroid provides a location, such as in x-ycoordinates, corresponding to the center of mass of the distribution inthe die. The geometric moments, such as a full set of 15 moments,generate an equivalent representation of the data set. The seven momentsof Hu comprise moments that are invariant under the actions oftranslation, scaling, and rotation.

The various features may be determined for any sets of data. In thepresent embodiment, the features are calculated based on bin data,outlier signature data, or other test data for wafers. Thus, the massmay generally correspond to the magnitude of the distribution of diewithin a bin of interest or other values, without giving any informationabout the location of the distribution. When the test value atcoordinates x and y is f(x,y), the mass M is suitably calculatedaccording to the following equation:

$M = \frac{\sum\limits_{x}{\sum\limits_{y}{f\left( {x,y} \right)}}}{N}$

where N is the total number of data points in the test data set. Themass is normalized so that there is consistency between data sets withdifferent numbers of data points, such as the number of die on a wafer.

The centroid may be defined by spatial coordinates, such as x_(c) andy_(c). The centroid provides location information by measuring thecenter of mass of the distribution of the die. The centroid of the datamay be calculated in any suitable manner, such as according to thefollowing equations:

${x_{c} = \frac{\sum\limits_{x}{\sum\limits_{y}{{xf}\left( {x,y} \right)}}}{\sum\limits_{x}{\sum\limits_{y}{f\left( {x,y} \right)}}}}{y_{c} = \frac{\sum\limits_{x}{\sum\limits_{y}{{yf}\left( {x,y} \right)}}}{\sum\limits_{x}{\sum\limits_{y}{f\left( {x,y} \right)}}}}$

The geometric moments of order (p=0 . . . 3, q=0 . . . 3) may becalculated according to the following equation:

$m_{pq} = \frac{\sum\limits_{x}{\sum\limits_{y}{x^{p}y^{q}{f\left( {x,y} \right)}}}}{\sum\limits_{x}{\sum\limits_{y}{f\left( {x,y} \right)}}}$

The information supplied by this set of moments provides an equivalentrepresentation of the data set, in the sense that the binmap can beconstructed from its moments of all orders. Thus, each momentcoefficient conveys a certain amount of the information residing in abinmap.

In the present embodiment, the seven moments of Hu are also considered(see Hu, M. K.,

“Visual Pattern recognition by moments invariants”, IRE Transactions onInformation Theory, Vol. 8(2), pp. 179-187, 1962). The seven moments ofHu are invariant under the actions of translation, scaling, androtation. The moments of Hu may be calculated according to the followingequations:Φ₁=η₂₀+η₀₂Φ₂=(η₂₀−η₀₂)²+4 η₁₁ ²Φ₃=(η₃₀−3η₁₂)²+(η₀₃−3η₂₁)²Φ₄=(η₃₀+η₁₂)²+(η₀₃+η₂₁)²Φ₅=(η₃₀−3η₁₂)(η₃₀+η₁₂)[(η₃₀+η₁₂)²−3(η₂₁+η₀₃)²]+(η₀₃−3η₂₁)(η₀₃+η₂₁)[(η₀₃+η₂₁)²−3(η₁₂+η₃₀)²]Φ₆=(η₂₀−η₀₂)[(η₃₀+η₁₂)²−(η₂₁+η₀₃)²]+4η₁₁(η₃₀+η₁₂)(η₀₃+η₂₁)Φ₇=(3η₂₁−η₀₃)(η₃₀+η₁₂)[(η₃₀+η₁₂)²−3(η₂₁+η₀₃)²]+(η₃₀−3η₁₂)(η₀₃+η₂₁)[(η₀₃+η₂₁)²−3(η₁₂+η₃₀)²]

where the η_(pq) are the central moments for all p, q defined asη_(pq)=ΣΣ(x−x _(c))^(p)(y−y _(c))^(q) f(x,y)

The first six of these moments are also invariant under the action ofreflection, while the last changes sign. The values of these quantitiescan be very large and different. To avoid precision problems, thelogarithms of the absolute values may be taken and passed on to theclassifier 2616 as features. The invariance of these features is anadvantage when binmaps or other data sets are analyzed with signatureclasses that do not depend on scale, location, or angular position.

A representative set of 25 features is suitably extracted from everybinmap or other data set designated by the pattern filter 2618. All or aportion of the features may be provided directly to the classifier 2616for classification. In one embodiment, the features are supplied to theneural network in order to select training data. In the presentembodiment, fewer than all of the features may be provided to theclassifier 2616, for example to reduce the number of features to beanalyzed and thus reduce the dimensionality of the analysis process.Reducing the number of features tends to reduce computational complexityand redundancy. In addition, the required generalization properties ofthe classifier 2616 may require the number of features to be limited.For example, for the present classifier 2616, the generalizationproperties of the classifier 2616 may correspond to the ratio of thenumber of training parameters N to the number of free classifierparameters. A larger number of features corresponds to a larger numberof classifier parameters, such as synaptic weights. For a finite andusually limited number N of training parameters, fewer features tend toimprove the generalization of the classifier 2616.

The feature selector 2622 of the present system analyzes the generatedfeatures according to selected criteria to select features forclassification. In particular, the feature selector 2622 is suitablyconfigured to select particular features to be analyzed and minimizeerrors that may be induced by providing fewer than all of the featuresto the classifier 2616. The feature selector 2622 may be configured inany suitable manner to select features for transfer to the classifier2616.

In the present embodiment, the feature selector 2622 implements agenetic algorithm to select the features. The genetic algorithm suitablyincludes a parallel search process, which tends to maintain multiplesolutions, eliminate the dubious solutions, and improve good solutions.The genetic algorithm analysis is suitably applied to the variousfeatures for a number of iterations, and the output of the algorithm isthe best solution found in the process of evolution.

Referring to FIG. 27 , to implement the genetic algorithm in the presentembodiment, the feature selector 2622 starts by initially definingvalues for the GA parameters (2710), starting a generation counter(2712), and randomly creating the initial population (2714). Thepopulation suitably comprises a collection of coded individuals, andeach individual represents a set of selected features. The sequence ofindividuals in the initial population is generated randomly, for exampleby an automatic computer program. Any suitable parameters may be used,such as parameters corresponding to a number of epochs, a number ofindividuals in a population, a size of chromosome, a cost function, aselection rate, a crossover/reproduction rate, and a mutation rate. Inthe present system, every population has ten different individuals,which represent the presence or not of a particular feature in theoptimal solution. In other words, each individual has been binaryencoded into a chromosome representing a set of features, i.e., a stringof 25 bits (number of features) where a “1” represents that particularfeature is taken into consideration for the classification and a “0”means that the feature in that position is not used.

The feature selector 2622 then evaluates the initial population (2716),applies crossover and mutation to the population (2718, 2720), andincrements the generation counter (2722). If the generation counter hasreached a preselected limit (2724), such as a maximum number ofgenerations, the feature selector 2622 terminates the analysis andprovides the selected features to the classifier 2616 (2726). If thelimit has not yet been reached, the feature selector 2622 repeatedlyevaluates the offspring population (2728) and applies the crossover andmutation to the population until the limit is reached.

The classifier 2616 classifies the identified patterns to differentknown categories or an unknown category. The classifier 2616 maycomprise any suitable classification system for classifying the patternsidentified by the pattern identifier 2614, such as Bayes or maximumlikelihood classifiers, supervised non-parametric classifiers, and/orsupervised or unsupervised rule-based classifiers. In one embodiment,the classifier 2616 may utilize a SOM to define the classes. In thisembodiment, the input data may be used to train the SOM. The number ofclasses may be a default number or may be selected by the user. Afterfeeding the input data into SOM, classes will automatically be formedbased upon the similarities between the various input data. Theclassifier 2616 of the present embodiment is configured to classify thepatterns based on analysis of the features selected by the featureselector 2622.

The classifier 2616 receives data for analysis, such as features whichhave been obtained by the feature extractor 2620 and selected by thefeature selector 2622. The data is processed to compare the data to datafor known patterns. If a known pattern matches the data, thecorresponding issue or characteristic for the known pattern is noted.For example, referring to FIG. 31A-B, the classifier 2616 may associateparticular characteristics in the input data with particular problemsources, such as by referring to a lookup table. The classifier 2616 mayalso assign a likelihood of a particular issue or characteristiccorresponding to the pattern. If no known pattern matches the data, thefailure to match is noted as well. The resulting information may then beprovided to the output element 208.

Referring to FIG. 32 , in the present embodiment, the classifier 2616comprises a two-stage classifier 3208. A first stage 3210 receives datafrom the individual data sources and classifies the data for potentialproblems, for example according to a set of rules or patterns in thedata. The second stage 3212 receives the data from the variousfirst-stage 3210 classifiers and combines the results to classify theoverall data to identify the most likely sources of problems or issuescharacterized by the data.

The classifiers in the two stages 3210, 3212 may comprise any suitablesystems for analyzing the data, such as intelligent systems, for exampleneural networks, particle swarm optimization (PSO) systems, geneticalgorithm (GA) systems, radial basic function (RBF) neural networks,multilayer perceptron (MLP) neural networks, RBF-PSO systems, MLP-PSOsystems, MLP-PSO-GA systems, SOM neural networks, or other types orcombinations of classifiers. The particular systems may be selectedaccording to the classifier's performance for the particular data set.

In one embodiment, the neural network second stage 3212 comprises a SOMand a feedforward neural network configured to analyze the featuresselected by the feature selector 2622. The feature values for thepatterned wafers detected with the feedforward neural network are usedas input into the SOM. The SOM according to various aspects of thepresent embodiment may be trained to function as a classifier. Thus,during the training of the SOM, test information containing variousoutput classes is provided.

The classifier 2616 may also provide a suggested corrective action basedon the corresponding characteristic. In particular, the classifier 2616may be configured to access a memory, such as the database 114, toidentify a set of corrective action candidates in response to variousfabrication and/or test process characteristics. For example, if thecharacteristic matching the identified pattern indicates that thecomponents have been excessively exposed to heat in a particular pointin the fabrication process, the classifier 2616 may check the database114 for potential corrective action corresponding to the characteristicto remedy the issue, such as to reduce the temperature or the durationof the exposure of the wafer at the particular fabrication point.

The pattern recognition system 2612 may also be configured to learnadditional information about the identified patterns and correspondingissues. For example, the pattern recognition system 2612 may beconfigured to receive diagnosis feedback information after identifying apattern. The diagnosis feedback information suitably corresponds toactual issues identified in the manufacturing or test process thatcaused the identified pattern. The pattern recognition system may thenuse the diagnosis feedback information for future analyses of data toidentify recurrence of the issues.

The various functions and elements of the present test system 100 may beconfigured to process multisite test data as well as conventionalsingle-site data. Unlike conventional single-site data, multisite datais received from different parts of the wafer simultaneously, and may bereceived using different hardware and software resources. Consequently,data from different sites may vary due to factors other than differencesin the devices, such as differences in the probe hardware. Accordingly,the test system 100 may be configured to analyze the test data formultisite testing to minimize potential problems associated withmultisite testing and/or identify problems that may be associated withmultisite testing.

For example, in one embodiment, the supplemental data analysis system206, the composite analysis system 214, and/or the diagnostic system 216may perform independent analyses on the test data for each individualsite, such that test data from each individual probe is treated as if itwere generated by a separate tester 102. Consequently, inconsistenciesbetween the various sites do not cause data analysis problems.

In another embodiment, the supplemental data analysis system 206, thecomposite analysis system 214, and/or the diagnostic system 216 mayanalyze the data from different sites independently for somecalculations, merge the data from two or more sites for othercalculations, and perform some calculations using both the independentsite data and the merged data. For example, statistical calculationsrelating to median tester values may be calculated independently foreach site. To perform proximity analysis, however, the test system 100may use a merged data set using data from all sites. The data may betreated in any suitable manner so that variations caused by multisitetesting are recognized and/or addressed.

In operation, data is received from the various sources. The data may beinitially analyzed for rules-based diagnoses, such as data correspondingexactly to know problems. The diagnosis system 216 generates an outputindicating the particular issues identified using the rules-basedanalysis. The data is then provided to the pattern recognition system2612, which analyzes the data to identify patterns in the data. Thepattern recognition system 2612 may utilize a SOM that has been trainedto recognize patterns in the data.

The particular implementations shown and described are merelyillustrative of the invention and its best mode and are not intended tootherwise limit the scope of the present invention in any way. For thesake of brevity, conventional signal processing, data transmission, andother functional aspects of the systems (and components of theindividual operating components of the systems) may not be described indetail. Furthermore, the connecting lines shown in the various figuresare intended to represent exemplary functional relationships and/orphysical couplings between the various elements. Many alternative oradditional functional relationships or physical connections may bepresent in a practical system. The present invention has been describedabove with reference to a preferred embodiment. Changes andmodifications may be made, however, without departing from the scope ofthe present invention. These and other changes or modifications areintended to be included within the scope of the present invention, asexpressed in the following claims.

The invention claimed is:
 1. A test data analysis system for analyzing test data generated by an automatic test equipment for a set of semiconductor components fabricated using a fabrication process and identifying a problem in the fabrication process, the system comprising: a processor configured to obtain the test data for the set of semiconductor components and to implement a diagnostic system; and a memory configured to store the test data; wherein the diagnostic system: filters the test data using a neural network configured to filter the test data, wherein the neural network comprises a self-organizing map; recognizes a pattern in the filtered test data corresponding to a recognized spatial pattern of the set of semiconductor components when the filtered test data is superimposed onto a wafermap; compares the recognized spatial pattern to a known spatial pattern associated with the problem; retrieves a stored neural network from the neural network library to analyze the filtered test data; classifies, using the stored neural network, the recognized spatial pattern according to the known spatial pattern, wherein the stored neural network comprises a self-organizing map and at least two stages, wherein: a first stage comprises a plurality of classifiers configured to receive different types of test data and generates first stage data based on the different types of test data; and a second stage configured to receive the first stage data from the plurality of classifiers and classify the known spatial pattern using the neural network based on the first stage data; identifies the problem in the fabrication process based on a classification of the recognized spatial pattern; and produces a report comprising the problem in the fabrication process.
 2. A test data analysis system according to claim 1, wherein the first and second stages comprise self-learning systems, and the first and second stages are independently trainable.
 3. A test data analysis system according to claim 1, wherein the diagnostic system comprises a classifier configured to classify the recognized spatial pattern according to the known spatial pattern.
 4. A test data analysis system according to claim 3, wherein the diagnostic system comprises a feature extractor configured to extract a feature from the test data associated with the recognized spatial pattern.
 5. A test data analysis system according to claim 4, wherein the feature extractor is configured to extract at least two features from the test data, and wherein the diagnostic system further comprises a feature selector configured to select fewer than all of the features for analysis. 